junghanacs🧠

Backlinks

#모음: #개발자 #생산성 #도구 #팁 #치트시트 #매뉴얼 #스크립트

95 min read

Table of Contents

BIBLIOGRAPHY

“Junghan0611/the-Book-of-Secret-Knowledge: A Collection of Inspiring Lists, Manuals, Cheatsheets, Blogs, Hacks, One-Liners, Cli/Web Tools and More.” 2024. https://github.com/junghan0611/the-book-of-secret-knowledge.

History

  • [2025-06-16 Mon 14:23] 노트로 만들만한 거야? 뭐가 많지? 헉 많네. 조직모드로 옮겨옴. 개발자
  • [2025-02-03 Mon 22:16] 리포지토리. 뭐가 많다.

소개: The Book of Secret Knowledge

What is it?

This repository is a collection of various materials and tools that I use every day in my work. It contains a lot of useful information gathered in one piece. It is an invaluable source of knowledge for me that I often look back on.

>

For whom?

For everyone, really. Here everyone can find their favourite tastes. But to be perfectly honest, it is aimed towards System and Network administrators, DevOps, Pentesters, and Security Researchers.

junghan0611/the-book-of-secret-knowledge

A collection of inspiring lists, manuals, cheatsheets, blogs, hacks, one-liners, cli/web tools and more. :2024: (“Junghan0611/the-Book-of-Secret-Knowledge: A Collection of Inspiring Lists, Manuals, Cheatsheets, Blogs, Hacks, One-Liners, Cli/Web Tools and More.” 2024)

CLI Tools

Shells

  • GNU Bash - is an sh-compatible shell that incorporates useful features from the Korn shell and C shell.
  • Zsh - is a shell designed for interactive use, although it is also a powerful scripting language.
  • tclsh - is a very powerful cross-platform shell, suitable for a huge range of uses.
  • bash-it - is a framework for using, developing and maintaining shell scripts and custom commands.
  • Oh My ZSH! - is the best framework for managing your Zsh configuration.
  • Oh My Fish - the Fishshell framework.
  • Starship - the cross-shell prompt written in Rust.
  • powerlevel10k - is a fast reimplementation of Powerlevel9k ZSH theme.

Shell plugins

  • z - tracks the folder you use the most and allow you to jump, without having to type the whole path.
  • fzf - is a general-purpose command-line fuzzy finder.
  • zsh-autosuggestions - Fish-like autosuggestions for Zsh.
  • zsh-syntax-highlighting - Fish shell like syntax highlighting for Zsh.
  • Awesome ZSH Plugins - A list of frameworks, plugins, themes and tutorials for ZSH.

Managers

  • Midnight Commander - is a visual file manager, licensed under GNU General Public License.
  • ranger - is a VIM-inspired filemanager for the console.
  • nnn - is a tiny, lightning fast, feature-packed file manager.
  • screen - is a full-screen window manager that multiplexes a physical terminal.
  • tmux - is a terminal multiplexer, lets you switch easily between several programs in one terminal.
  • tmux-cssh - is a tool to set comfortable and easy to use functionality tmux-sessions.

Text editors

  • vi - is one of the most common text editors on Unix.
  • vim - is a highly configurable text editor.
  • emacs - is an extensible, customizable, free/libre text editor, and more.
  • micro - is a modern and intuitive terminal-based text editor.
  • neovim - is a free open source, powerful, extensible and usable code editor.
  • spacemacs - a community-driven Emacs distribution.
  • spacevim - a community-driven vim distribution.

Files and directories

  • fd - is a simple, fast and user-friendly alternative to find.
  • ncdu - is an easy to use, fast disk usage analyzer.

Network

  • PuTTY - is an SSH and telnet client, developed originally by Simon Tatham.
  • Mosh - is a SSH wrapper designed to keep a SSH session alive over a volatile connection.
  • Eternal Terminal - enables mouse-scrolling and tmux commands inside the SSH session.
  • nmap - is a free and open source (license) utility for network discovery and security auditing.
  • zmap - is a fast single packet network scanner designed for Internet-wide network surveys.
  • Rust Scan - to find all open ports faster than Nmap.
  • masscan - is the fastest Internet port scanner, spews SYN packets asynchronously.
  • pbscan - is a faster and more efficient stateless SYN scanner and banner grabber.
  • hping - is a command-line oriented TCP/IP packet assembler/analyzer.
  • mtr - is a tool that combines the functionality of the ‘traceroute’ and ‘ping’ programs in a single tool.
  • mylg - utility which combines the functions of the different network probes in one diagnostic tool.
  • netcat - utility which reads and writes data across network connections, using the TCP/IP protocol.
  • socat - utility which transfers data between two objects.
  • tcpdump - is a powerful command-line packet analyzer.
  • tshark - is a tool that allows us to dump and analyze network traffic (wireshark cli).
  • Termshark - is a simple terminal user-interface for tshark.
  • ngrep - is like GNU grep applied to the network layer.
  • netsniff-ng - is a Swiss army knife for your daily Linux network plumbing if you will.
  • sockdump - dump unix domain socket traffic.
  • stenographer - is a packet capture solution which aims to quickly spool all packets to disk.
  • tcpterm - visualize packets in TUI.
  • bmon - is a monitoring and debugging tool to capture networking related statistics and prepare them visually.
  • iptraf-ng - is a console-based network monitoring program for Linux that displays information about IP traffic.
  • vnstat - is a network traffic monitor for Linux and BSD.
  • iPerf3 - is a tool for active measurements of the maximum achievable bandwidth on IP networks.
  • ethr - is a Network Performance Measurement Tool for TCP, UDP & HTTP.
  • Etherate - is a Linux CLI based Ethernet and MPLS traffic testing tool.
  • echoip - is a IP address lookup service.
  • Nemesis - packet manipulation CLI tool; craft and inject packets of several protocols.
  • packetfu - a mid-level packet manipulation library for Ruby.
  • Scapy - packet manipulation library; forge, send, decode, capture packets of a wide number of protocols.
  • impacket - is a collection of Python classes for working with network protocols.
  • ssh-audit - is a tool for SSH server auditing.
  • aria2 - is a lightweight multi-protocol & multi-source command-line download utility.
  • iptables-tracer - observe the path of packets through the iptables chains.
  • inception - a highly configurable tool to check for whatever you like against any number of hosts.
  • mRemoteNG - a fork of mRemote, multi-tabbed PuTTy on steroids!

Network (DNS)

  • dnsdiag - is a DNS diagnostics and performance measurement tools.
  • fierce - is a DNS reconnaissance tool for locating non-contiguous IP space.
  • subfinder - is a subdomain discovery tool that discovers valid subdomains for websites.
  • sublist3r - is a fast subdomains enumeration tool for penetration testers.
  • amass - is tool that obtains subdomain names by scraping data sources, crawling web archives, and more.
  • namebench - provides personalized DNS server recommendations based on your browsing history.
  • massdns - is a high-performance DNS stub resolver for bulk lookups and reconnaissance.
  • knock - is a tool to enumerate subdomains on a target domain through a wordlist.
  • dnsperf - DNS performance testing tools.
  • dnscrypt-proxy 2 - a flexible DNS proxy, with support for encrypted DNS protocols.
  • dnsdbq - API client providing access to passive DNS database systems.
  • grimd - fast dns proxy, built to black-hole internet advertisements and malware servers.
  • dnstwist - detect typosquatters, phishing attacks, fraud, and brand impersonation.

Network (HTTP)

  • curl - is a command line tool and library for transferring data with URLs.
  • kurly - is an alternative to the widely popular curl program, written in Golang.
  • HTTPie - is an user-friendly HTTP client.
  • wuzz - is an interactive cli tool for HTTP inspection.
  • h2spec - is a conformance testing tool for HTTP/2 implementation.
  • h2t - is a simple tool to help sysadmins to hardening their websites.
  • htrace.sh - is a simple Swiss Army knife for http/https troubleshooting and profiling.
  • httpstat - is a tool that visualizes curl statistics in a way of beauty and clarity.
  • httplab - is an interactive web server.
  • Lynx - is a text browser for the World Wide Web.
  • Browsh - is a fully interactive, real-time, and modern text-based browser.
  • HeadlessBrowsers - a list of (almost) all headless web browsers in existence.
  • ab - is a single-threaded command line tool for measuring the performance of HTTP web servers.
  • siege - is an http load testing and benchmarking utility.
  • wrk - is a modern HTTP benchmarking tool capable of generating significant load.
  • wrk2 - is a constant throughput, correct latency recording variant of wrk.
  • vegeta - is a constant throughput, correct latency recording variant of wrk.
  • bombardier - is a fast cross-platform HTTP benchmarking tool written in Go.
  • gobench - http/https load testing and benchmarking tool.
  • hey - HTTP load generator, ApacheBench (ab) replacement, formerly known as rakyll/boom.
  • boom - is a script you can use to quickly smoke-test your web app deployment.
  • SlowHTTPTest - is a tool that simulates some Application Layer Denial of Service attacks by prolonging HTTP.
  • gobuster - is a free and open source directory/file & DNS busting tool written in Go.
  • ssllabs-scan - command-line reference-implementation client for SSL Labs APIs.
  • http-observatory - Mozilla HTTP Observatory cli version.
  • Hurl - is a command line tool to run and test HTTP requests with plain text.

SSL

  • openssl - is a robust, commercial-grade, and full-featured toolkit for the TLS and SSL protocols.
  • gnutls-cli - client program to set up a TLS connection to some other computer.
  • sslyze \- fast and powerful SSL/TLS server scanning library.
  • sslscan - tests SSL/TLS enabled services to discover supported cipher suites.
  • testssl.sh - testing TLS/SSL encryption anywhere on any port.
  • cipherscan - a very simple way to find out which SSL ciphersuites are supported by a target.
  • spiped - is a utility for creating symmetrically encrypted and authenticated pipes between socket addresses.
  • Certbot - is EFF’s tool to obtain certs from Let’s Encrypt and (optionally) auto-enable HTTPS on your server.
  • mkcert - simple zero-config tool to make locally trusted development certificates with any names you’d like.
  • certstrap - tools to bootstrap CAs, certificate requests, and signed certificates.
  • Sublert - is a security and reconnaissance tool to automatically monitor new subdomains.
  • mkchain - open source tool to help you build a valid SSL certificate chain.
  • ssl-cert-check - SSL Certification Expiration Checker.

Security

  • SELinux - provides a flexible Mandatory Access Control (MAC) system built into the Linux kernel.
  • AppArmor - proactively protects the operating system and applications from external or internal threats.
  • grapheneX - Automated System Hardening Framework.
  • DevSec Hardening Framework - Security + DevOps: Automatic Server Hardening.

Auditing Tools

  • ossec - actively monitoring all aspects of system activity with file integrity monitoring.
  • auditd - provides a way to track security-relevant information on your system.
  • Tiger - is a security tool that can be use both as a security audit and intrusion detection system.
  • Lynis - battle-tested security tool for systems running Linux, macOS, or Unix-based operating system.
  • LinEnum - scripted Local Linux Enumeration & Privilege Escalation Checks.
  • Rkhunter - scanner tool for Linux systems that scans backdoors, rootkits and local exploits on your systems.
  • PE-sieve - is a light-weight tool that helps to detect malware running on the system.
  • PEASS - privilege escalation tools for Windows and Linux/Unix and MacOS.

System Diagnostics/Debuggers

  • strace - diagnostic, debugging and instructional userspace utility for Linux.
  • DTrace - is a performance analysis and troubleshooting tool.
  • ltrace - is a library call tracer, used to trace calls made by programs to library functions.
  • ptrace-burrito - is a friendly wrapper around ptrace.
  • perf-tools - performance analysis tools based on Linux perf\_events (aka perf) and ftrace.
  • bpftrace - high-level tracing language for Linux eBPF.
  • sysdig - system exploration and troubleshooting tool with first class support for containers.
  • Valgrind - is an instrumentation framework for building dynamic analysis tools.
  • gperftools - high-performance multi-threaded malloc() implementation, plus some performance analysis tools.
  • glances - cross-platform system monitoring tool written in Python.
  • htop - interactive text-mode process viewer for Unix systems. It aims to be a better ‘top’.
  • bashtop - Linux resource monitor written in pure Bash.
  • nmon - a single executable for performance monitoring and data analysis.
  • atop - ASCII performance monitor. Includes statistics for CPU, memory, disk, swap, network, and processes.
  • lsof - displays in its output information about files that are opened by processes.
  • FlameGraph - stack trace visualizer.
  • lsofgraph - convert Unix lsof output to a graph showing FIFO and UNIX interprocess communication.
  • rr - is a lightweight tool for recording, replaying and debugging execution of applications.
  • Performance Co-Pilot - a system performance analysis toolkit.
  • hexyl - a command-line hex viewer.
  • Austin - Python frame stack sampler for CPython.

Log Analyzers

  • angle-grinder - slice and dice log files on the command line.
  • lnav - log file navigator with search and automatic refresh.
  • GoAccess - real-time web log analyzer and interactive viewer that runs in a terminal.
  • ngxtop - real-time metrics for nginx server.

Databases

  • usql - universal command-line interface for SQL databases.
  • pgcli - postgres CLI with autocompletion and syntax highlighting.
  • mycli - terminal client for MySQL with autocompletion and syntax highlighting.
  • litecli - SQLite CLI with autocompletion and syntax highlighting.
  • mssql-cli - SQL Server CLI with autocompletion and syntax highlighting.
  • OSQuery - is a SQL powered operating system instrumentation, monitoring, and analytics framework.
  • pgsync - sync data from one Postgres database to another.
  • iredis - a terminal client for redis with autocompletion and syntax highlighting.
  • SchemaCrawler - generates an E-R diagram of your database.

TOR

  • Nipe - script to make Tor Network your default gateway.
  • multitor - a tool that lets you create multiple TOR instances with a load-balancing.

Messengers/IRC Clients

  • Irssi - is a free open source terminal based IRC client.
  • WeeChat - is an extremely extensible and lightweight IRC client.

Productivity

Other

  • sysadmin-util - tools for Linux/Unix sysadmins.
  • incron - is an inode-based filesystem notification technology.
  • lsyncd - synchronizes local directories with remote targets (Live Syncing Daemon).
  • GRV - is a terminal based interface for viewing Git repositories.
  • Tig - text-mode interface for Git.
  • tldr - simplified and community-driven man pages.
  • archiver - easily create and extract .zip, .tar, .tar.gz, .tar.bz2, .tar.xz, .tar.lz4, .tar.sz, and .rar.
  • commander.js - minimal CLI creator in JavaScript.
  • gron - make JSON greppable!
  • bed - binary editor written in Go.

GUI Tools

Terminal emulators

  • Guake - is a dropdown terminal made for the GNOME desktop environment.
  • Terminator - is based on GNOME Terminal, useful features for sysadmins and other users.
  • Kitty - is a GPU based terminal emulator that supports smooth scrolling and images.
  • Alacritty - is a fast, cross-platform, OpenGL terminal emulator.

Network

  • Wireshark - is the world’s foremost and widely-used network protocol analyzer.
  • Ettercap - is a comprehensive network monitor tool.
  • EtherApe - is a graphical network monitoring solution.
  • Packet Sender - is a networking utility for packet generation and built-in UDP/TCP/SSL client and servers.
  • Ostinato - is a packet crafter and traffic generator.
  • JMeter™ - open source software to load test functional behavior and measure performance.
  • locust - scalable user load testing tool written in Python.

Browsers

  • TOR Browser - protect your privacy and defend yourself against network surveillance and traffic analysis.

Password Managers

  • KeePassXC - store your passwords safely and auto-type them into your everyday websites and apps.
  • Bitwarden - open source password manager with built-in sync.
  • Vaultwarden - unofficial Bitwarden compatible server written in Rust.

Messengers/IRC Clients

  • HexChat - is an IRC client based on XChat.
  • Pidgin - is an easy to use and free chat client used by millions.

Messengers (end-to-end encryption)

  • Signal - is an encrypted communications app.
  • Wire - secure messaging, file sharing, voice calls and video conferences.
  • TorChat - decentralized anonymous instant messenger on top of Tor Hidden Services.
  • Matrix - an open network for secure, decentralized, real-time communication.

Text editors

  • Sublime Text - is a lightweight, cross-platform code editor known for its speed, ease of use.
  • Visual Studio Code - an open-source and free source code editor developed by Microsoft.
  • Atom - a hackable text editor for the 21st Century.

Web Tools

Browsers

SSL/Security

  • SSLLabs Server Test - performs a deep analysis of the configuration of any SSL web server.
  • SSLLabs Server Test (DEV) - performs a deep analysis of the configuration of any SSL web server.
  • ImmuniWeb® SSLScan - test SSL/TLS (PCI DSS, HIPAA and NIST).
  • SSL Check - scan your website for non-secure content.
  • SSL Scanner - analyze website security.
  • CryptCheck - test your TLS server configuration (e.g. ciphers).
  • urlscan.io - service to scan and analyse websites.
  • Report URI - monitoring security policies like CSP and HPKP.
  • CSP Evaluator - allows developers and security experts to check if a Content Security Policy.
  • Useless CSP - public list about CSP in some big players (might make them care a bit more).
  • Why No HTTPS? - top 100 websites by Alexa rank not automatically redirecting insecure requests.
  • TLS Cipher Suite Search\- cipher suite search engine.
  • cipherli.st - strong ciphers for Apache, Nginx, Lighttpd, and more. \*
  • dhtool - public Diffie-Hellman parameter service/tool.
  • badssl.com - memorable site for testing clients against bad SSL configs.
  • tlsfun.de - registered for various tests regarding the TLS/SSL protocol.
  • CAA Record Helper - generate a CAA policy.
  • Common CA Database - repository of information about CAs, and their root and intermediate certificates.
  • CERTSTREAM - real-time certificate transparency log update stream.
  • crt.sh - discovers certificates by continually monitoring all of the publicly known CT.
  • Hardenize - deploy the security standards.
  • Cipher suite compatibility - test TLS cipher suite compatibility.
  • urlvoid - this service helps you detect potentially malicious websites.
  • security.txt - a proposed standard (generator) which allows websites to define security policies.
  • ssl-config-generator - help you follow the Mozilla Server Side TLS configuration guidelines.
  • TLScan - pure python, SSL/TLS protocol and cipher scanner/enumerator.

HTTP Headers & Web Linters

  • Security Headers - analyse the HTTP response headers (with rating system to the results).
  • Observatory by Mozilla - set of tools to analyze your website.
  • webhint - is a linting tool that will help you with your site’s accessibility, speed, security, and more.

DNS

  • ViewDNS - one source for free DNS related tools and information.
  • DNSLookup - is an advanced DNS lookup tool.
  • DNSlytics - online DNS investigation tool.
  • DNS Spy - monitor, validate and verify your DNS configurations.
  • Zonemaster - helps you to control how your DNS works.
  • Leaf DNS - comprehensive DNS tester.
  • Find subdomains online - find subdomains for security assessment penetration test.
  • DNSdumpster - dns recon & research, find & lookup dns records.
  • DNS Table online - search for DNS records by domain, IP, CIDR, ISP.
  • intoDNS - DNS and mail server health checker.
  • DNS Bajaj - check the delegation of your domain.
  • BuddyDNS Delegation LAB - check, trace and visualize delegation of your domain.
  • dnssec-debugger - DS or DNSKEY records validator.
  • PTRarchive.com - this site is responsible for the safekeeping of historical reverse DNS records.
  • xip.io - wildcard DNS for everyone.
  • nip.io - dead simple wildcard DNS for any IP Address.
  • dnslookup (ceipam) - one of the best DNS propagation checker (and not only).
  • What’s My DNS - DNS propagation checking tool.
  • DNSGrep - quickly searching large DNS datasets.

Mail

Encoders/Decoders and Regex testing

  • URL Encode/Decode - tool from above to either encode or decode a string of text.
  • Uncoder - the online translator for search queries on log data.
  • Regex101 - online regex tester and debugger: PHP, PCRE, Python, Golang and JavaScript.
  • RegExr - online tool to learn, build, & test Regular Expressions (RegEx / RegExp).
  • RegEx Testing - online regex testing tool.
  • RegEx Pal - online regex testing tool + other tools.
  • The Cyber Swiss Army Knife - a web app for encryption, encoding, compression and data analysis.

Net-tools

  • Netcraft - detailed report about the site, helping you to make informed choices about their integrity. \*
  • RIPE NCC Atlas - a global, open, distributed Internet measurement platform.
  • Robtex - uses various sources to gather public information about IP numbers, domain names, host names, etc.
  • Security Trails - APIs for Security Companies, Researchers and Teams.
  • Online Curl - curl test, analyze HTTP Response Headers.
  • Online Tools for Developers - HTTP API tools, testers, encoders, converters, formatters, and other tools.
  • Ping.eu - online Ping, Traceroute, DNS lookup, WHOIS and others.
  • Network-Tools - network tools for webmasters, IT technicians & geeks.
  • BGPview - search for any ASN, IP, Prefix or Resource name.
  • Is BGP safe yet? - check BGP (RPKI) security of ISPs and other major Internet players.
  • Riseup - provides online communication tools for people and groups working on liberatory social change.
  • VirusTotal - analyze suspicious files and URLs to detect types of malware.

Privacy

Code parsers/playgrounds

  • ShellCheck - finds bugs in your shell scripts.
  • explainshell - get interactive help texts for shell commands.
  • jsbin - live pastebin for HTML, CSS & JavaScript, and more.
  • CodeSandbox - online code editor for web application development.
  • PHP Sandbox - test your PHP code with this code tester.
  • Repl.it - an instant IDE to learn, build, collaborate, and host all in one place.
  • vclFiddle - is an online tool for experimenting with the Varnish Cache VCL.
  • Haskell Dockerfile Linter - a smarter Dockerfile linter that helps you build best practice Docker images.

Performance

  • GTmetrix - analyze your site’s speed and make it faster.
  • Sucuri loadtimetester - test here the performance of any of your sites from across the globe.
  • Pingdom Tools - analyze your site’s speed around the world.
  • PingMe.io - run website latency tests across multiple geographic regions.
  • PageSpeed Insights - analyze your site’s speed and make it faster.
  • web.dev - helps developers like you learn and apply the web’s modern capabilities to your own sites and apps.
  • Lighthouse - automated auditing, performance metrics, and best practices for the web.

Mass scanners (search engines)

  • Censys - platform that helps information security practitioners discover, monitor, and analyze devices.
  • Shodan - the world’s first search engine for Internet-connected devices.
  • Shodan 2000 - this tool looks for randomly generated data from Shodan.
  • GreyNoise - mass scanner such as Shodan and Censys.
  • ZoomEye - search engine for cyberspace that lets the user find specific network components.
  • netograph - tools to monitor and understand deep structure of the web.
  • FOFA - is a cyberspace search engine.
  • onyphe - is a search engine for open-source and cyber threat intelligence data collected.
  • IntelligenceX - is a search engine and data archive.
  • binaryedge - it scan the entire internet space and create real-time threat intelligence streams and reports.
  • Spyse - Internet assets registry: networks, threats, web objects, etc.
  • wigle - is a submission-based catalog of wireless networks. All the networks. Found by Everyone.
  • PublicWWW - find any alphanumeric snippet, signature or keyword in the web pages HTML, JS and CSS code.
  • IntelTechniques - this repository contains hundreds of online search utilities.
  • hunter - lets you find email addresses in seconds and connect with the people that matter for your business.
  • GhostProject? - search by full email address or username.
  • databreaches - was my email affected by data breach?
  • We Leak Info - world’s fastest and largest data breach search engine.
  • Pulsedive - scans of malicious URLs, IPs, and domains, including port scans and web requests.
  • Buckets by Grayhatwarfar - database with public search for Open Amazon S3 Buckets and their contents.
  • Vigilante.pw - the breached database directory.
  • builtwith - find out what websites are built with.
  • NerdyData - search the web’s source code for technologies, across millions of sites.
  • zorexeye - search for sites, images, apps, softwares & more.
  • Mamont’s open FTP Index - if a target has an open FTP site with accessible content it will be listed here.
  • OSINT Framework - focused on gathering information from free tools or resources.
  • maltiverse - is a service oriented to cybersecurity analysts.
  • Leaked Source - is a collaboration of data found online in the form of a lookup.
  • We Leak Info - to help everyday individuals secure their online life, avoiding getting hacked.
  • pipl - is the place to find the person behind the email address, social username or phone number.
  • abuse.ch - is operated by a random swiss guy fighting malware for non-profit.
  • malc0de - malware search engine.
  • Cybercrime Tracker - monitors and tracks various malware families that are used to perpetrate cyber crimes.
  • shhgit - find GitHub secrets in real time.
  • searchcode - helping you find real world examples of functions, API’s and libraries.
  • Insecam - the world biggest directory of online surveillance security cameras.
  • index-of - contains great stuff like: security, hacking, reverse engineering, cryptography, programming etc.
  • Rapid7 Labs Open Data - is a great resources of datasets from Project Sonar.
  • Common Response Headers - the largest database of HTTP response headers.
  • InQuest Labs - InQuest Labs is an open, interactive, and API driven data portal for security researchers.

Generators

Passwords

  • have i been pwned? - check if you have an account that has been compromised in a data breach.
  • dehashed - is a hacked database search engine.
  • Leaked Source - is a collaboration of data found online in the form of a lookup.

CVE/Exploits databases

  • CVE Mitre - list of publicly known cybersecurity vulnerabilities.
  • CVE Details - CVE security vulnerability advanced database.
  • Exploit DB - CVE compliant archive of public exploits and corresponding vulnerable software.
  • 0day.today - exploits market provides you the possibility to buy/sell zero-day exploits.
  • sploitus - the exploit and tools database.
  • cxsecurity - free vulnerability database.
  • Vulncode-DB - is a database for vulnerabilities and their corresponding source code if available.
  • cveapi - free API for CVE data.

Mobile apps scanners

  • ImmuniWeb® Mobile App Scanner - test security and privacy of mobile apps (iOS & Android).
  • Quixxi - free Mobile App Vulnerability Scanner for Android & iOS.
  • Ostorlab - analyzes mobile application to identify vulnerabilities and potential weaknesses.

Private Search Engines

  • Startpage - the world’s most private search engine.
  • searX - a privacy-respecting, hackable metasearch engine.
  • darksearch - the 1st real Dark Web search engine.
  • Qwant - the search engine that respects your privacy.
  • DuckDuckGo - the search engine that doesn’t track you.
  • Swisscows - privacy safe web search
  • Disconnect - the search engine that anonymizes your searches.
  • MetaGer - the search engine that uses anonymous proxy and hidden Tor branches.

Secure Webmail Providers

  • CounterMail - online email service, designed to provide maximum security and privacy.
  • Mail2Tor - is a Tor Hidden Service that allows anyone to send and receive emails anonymously.
  • Tutanota - is the world’s most secure email service and amazingly easy to use.
  • Protonmail - is the world’s largest secure email service, developed by CERN and MIT scientists.
  • Startmail - private & encrypted email made easy.

Crypto

  • Keybase - it’s open source and powered by public-key cryptography.

PGP Keyservers

Systems/Services

Operating Systems

  • Slackware - the most “Unix-like” Linux distribution.
  • OpenBSD - multi-platform 4.4BSD-based UNIX-like operating system.
  • HardenedBSD - HardenedBSD aims to implement innovative exploit mitigation and security solutions.
  • Kali Linux - Linux distribution used for Penetration Testing, Ethical Hacking and network security assessments.
  • Parrot Security OS - cyber security GNU/Linux environment.
  • Backbox Linux - penetration test and security assessment oriented Ubuntu-based Linux distribution.
  • BlackArch - is an Arch Linux-based penetration testing distribution for penetration testers.
  • Pentoo - is a security-focused livecd based on Gentoo.
  • Security Onion - Linux distro for intrusion detection, enterprise security monitoring, and log management.
  • Tails - is a live system that aims to preserve your privacy and anonymity.
  • vedetta - OpenBSD router boilerplate.
  • Qubes OS - is a security-oriented OS that uses Xen-based virtualization.

HTTP(s) Services

  • Varnish Cache - HTTP accelerator designed for content-heavy dynamic web sites.
  • Nginx - open source web and reverse proxy server that is similar to Apache, but very light weight.
  • OpenResty - is a dynamic web platform based on NGINX and LuaJIT.
  • Tengine - a distribution of Nginx with some advanced features.
  • Caddy Server - is an open source, HTTP/2-enabled web server with HTTPS by default.
  • HAProxy - the reliable, high performance TCP/HTTP load balancer.

DNS Services

  • Unbound - validating, recursive, and caching DNS resolver (with TLS).
  • Knot Resolver - caching full resolver implementation, including both a resolver library and a daemon.
  • PowerDNS - is an open source authoritative DNS server, written in C++ and licensed under the GPL.

Other Services

  • 3proxy - tiny free proxy server.

Security/hardening

  • Emerald Onion - is a 501(c)(3) nonprofit organization and transit internet service provider (ISP).
  • pi-hole - the Pi-hole® is a DNS sinkhole that protects your devices from unwanted content.
  • maltrail - malicious traffic detection system.
  • security\_monkey - monitors AWS, GCP, OpenStack, and GitHub orgs for assets and their changes over time.
  • firecracker - secure and fast microVMs for serverless computing.
  • streisand - sets up a new server running your choice of WireGuard, OpenSSH, OpenVPN, and more.

Networks

Tools

  • CapAnalysis - web visual tool to analyze large amounts of captured network traffic (PCAP analyzer).
  • netbox - IP address management (IPAM) and data center infrastructure management (DCIM) tool.

Labs

  • NRE Labs - learn automation by doing it. Right now, right here, in your browser.

Other

Containers/Orchestration

CLI Tools

  • gvisor - container runtime sandbox.
  • ctop - top-like interface for container metrics.

Web Tools

  • Moby - a collaborative project for the container ecosystem to assemble container-based system.
  • Traefik - open source reverse proxy/load balancer provides easier integration with Docker and Let’s encrypt.
  • kong - The Cloud-Native API Gateway.
  • rancher - complete container management platform.
  • portainer - making Docker management easy.
  • nginx-proxy - automated nginx proxy for Docker containers using docker-gen.
  • bunkerized-nginx - nginx docker image “secure by default”.

Security

  • docker-bench-security - checks for dozens of common best-practices around deploying Docker.
  • trivy - vulnerability scanner for containers, suitable for CI.
  • Harbor - cloud native registry project that stores, signs, and scans content.
  • Houdini - hundreds of offensive and useful docker images for network intrusion.

Manuals/Tutorials/Best Practices

Manuals/Howtos/Tutorials

Shell/Command line

Text Editors

Python

Sed & Awk & Other

  • F’Awk Yeah! - advanced sed and awk usage (Parsing for Pentesters 3).

\*nix & Network

Microsoft

  • AD-Attack-Defense - attack and defend active directory using modern post exploitation activity.

Large-scale systems

System hardening

Security & Privacy

Web Apps

All-in-one

Ebooks

Other

Inspiring Lists

SysOps/DevOps

Developers

Security/Pentesting

Other

Blogs/Podcasts/Videos

SysOps/DevOps

Developers

Geeky Persons

  • Brendan Gregg’s Blog - is an industry expert in computing performance and cloud computing.
  • Gynvael “GynDream” Coldwind - is a IT security engineer at Google.
  • Michał “lcamtuf” Zalewski - white hat hacker, computer security expert.
  • Mattias Geniar - developer, sysadmin, blogger, podcaster and public speaker.
  • Nick Craver - software developer and systems administrator for Stack Exchange.
  • Scott Helme - security researcher, speaker and founder of securityheaders.com and report-uri.com.
  • Brian Krebs - The Washington Post and now an Independent investigative journalist.
  • Bruce Schneier - is an internationally renowned security technologist, called a “security guru”.
  • Chrissy Morgan - advocate of practical learning, Chrissy also takes part in bug bounty programs.
  • Andy Gill - is a hacker at heart who works as a senior penetration tester.
  • Daniel Miessler - cybersecurity expert and writer.
  • Samy Kamkar - is an American privacy and security researcher, computer hacker.
  • Javvad Malik - is a security advocate at AlienVault, a blogger event speaker and industry commentator.
  • Graham Cluley - public speaker and independent computer security analyst.
  • Kacper Szurek - detection engineer at ESET.
  • Troy Hunt - web security expert known for public education and outreach on security topics.
  • raymii.org - sysadmin specializing in building high availability cloud environments.
  • Robert Penz - IT security expert.

Geeky Blogs

  • Linux Audit - the Linux security blog about auditing, hardening and compliance by Michael Boelen.
  • Linux Security Expert - trainings, howtos, checklists, security tools, and more.
  • The Grymoire - collection of useful incantations for wizards, be you computer wizards, magicians, or whatever.
  • Secjuice - is the only non-profit, independent and volunteer led publication in the information security space.
  • Decipher - security news that informs and inspires.

Geeky Vendor Blogs

  • Tenable Podcast - conversations and interviews related to Cyber Exposure, and more.
  • Sophos - threat news room, giving you news, opinion, advice and research on computer security issues.
  • Tripwire State of Security - blog featuring the latest news, trends and insights on current security issues.
  • Malwarebytes Labs Blog - security blog aims to provide insider news about cybersecurity.
  • TrustedSec - latest news, and trends about cybersecurity.
  • PortSwigger Web Security Blog - about web app security vulns and top tips from our team of web security.
  • AT&T Cybersecurity blog - news on emerging threats and practical advice to simplify threat detection.
  • Thycotic - where CISOs and IT Admins come to learn about industry trends, IT security, and more.

Geeky Cybersecurity Podcasts

Geeky Cybersecurity Video Blogs

  • rev3rse security - offensive, binary exploitation, web app security, hardening, red team, blue team.
  • LiveOverflow - a lot more advanced topics than what is typically offered in paid online courses - but for free.
  • J4vv4D - the important information regarding our internet security.
  • CyberTalks - talks, interviews, and article about cybersecurity.

Best Personal Twitter Accounts

  • @blackroomsec - a white-hat hacker/pentester. Intergalactic Minesweeper Champion 1990.
  • @MarcoCiappelli - Co-Founder @ITSPmagazine, at the intersection of IT security and society.
  • @binitamshah - Linux Evangelist. Malwares. Kernel Dev. Security Enthusiast.
  • @joe\_carson - an InfoSec Professional and Tech Geek.
  • @mikko - CRO at F-Secure, Reverse Engineer, TED Speaker, Supervillain.
  • @esrtweet - often referred to as ESR, is an American software developer, and open-source software advocate.
  • @gynvael - security researcher/programmer, @DragonSectorCTF founder/player, technical streamer.
  • @x0rz - Security Researcher & Cyber Observer.
  • @hasherezade - programmer, malware analyst. Author of PEbear, PEsieve, libPeConv.
  • @TinkerSec - tinkerer, cypherpunk, hacker.
  • @alisaesage - independent hacker and researcher.
  • @SwiftOnSecurity - systems security, industrial safety, sysadmin, author of decentsecurity.com.
  • @dakami - is one of just seven people with the authority to restore the DNS root keys.
  • @samykamkar - is a famous “grey hat” hacker, security researcher, creator of the MySpace “Samy” worm.
  • @securityweekly - founder & CTO of Security Weekly podcast network.
  • @jack\_daniel - @SecurityBSides co-founder.
  • @thegrugq - Security Researcher.
  • @matthew\_d\_green - a cryptographer and professor at Johns Hopkins University.

Best Commercial Twitter Accounts

  • @haveibeenpwned - check if you have an account that has been compromised in a data breach.
  • @bugcrowd - trusted by more of the Fortune 500 than any other crowdsourced security platform.
  • @Malwarebytes - most trusted security company. Unmatched threat visibility.
  • @sansforensics - the world’s leading Digital Forensics and Incident Response provider.
  • @attcyber - AT&T Cybersecurity’s Edge-to-Edge technologies provide threat intelligence, and more.
  • @TheManyHatsClub - an information security focused podcast and group of individuals from all walks of life.
  • @hedgehogsec - Hedgehog Cyber. Gibraltar and Manchester’s top boutique information security firm.
  • @NCSC - the National Cyber Security Centre. Helping to make the UK the safest place to live and work online.
  • @Synacktiv - IT security experts.

A piece of history

Other

Hacking/Penetration Testing

Pentesters arsenal tools

  • Sandcat Browser - a penetration-oriented browser with plenty of advanced functionality already built in.
  • Metasploit - tool and framework for pentesting system, web and many more.
  • Burp Suite - tool for testing web app security, intercepting proxy to replay, inject, scan and fuzz.
  • OWASP Zed Attack Proxy - intercepting proxy to replay, inject, scan and fuzz HTTP requests.
  • w3af - is a Web Application Attack and Audit Framework.
  • mitmproxy - an interactive TLS-capable intercepting HTTP proxy for penetration testers.
  • Nikto2 - web server scanner which performs comprehensive tests against web servers for multiple items.
  • sqlmap - tool that automates the process of detecting and exploiting SQL injection flaws.
  • Recon-ng - is a full-featured Web Reconnaissance framework written in Python.
  • AutoRecon - is a network reconnaissance tool which performs automated enumeration of services.
  • Faraday - an Integrated Multiuser Pentest Environment.
  • Photon - incredibly fast crawler designed for OSINT.
  • XSStrike - most advanced XSS detection suite.
  • Sn1per - automated pentest framework for offensive security experts.
  • vuls - is an agent-less vulnerability scanner for Linux, FreeBSD, and other.
  • tsunami - is a general purpose network security scanner with an extensible plugin system.
  • aquatone - a tool for domain flyovers.
  • BillCipher - information gathering tool for a website or IP address.
  • WhatWaf - detect and bypass web application firewalls and protection systems.
  • Corsy - CORS misconfiguration scanner.
  • Raccoon - is a high performance offensive security tool for reconnaissance and vulnerability scanning.
  • dirhunt - find web directories without bruteforce.
  • John The Ripper - is a fast password cracker, currently available for many flavors of Unix, Windows, and other.
  • hashcat - world’s fastest and most advanced password recovery utility.
  • p0f - is a tool to identify the players behind any incidental TCP/IP communications.
  • ssh\_scan - a prototype SSH configuration and policy scanner.
  • LeakLooker - find open databases - powered by Binaryedge.io
  • exploitdb - searchable archive from The Exploit Database.
  • getsploit - is a command line utility for searching and downloading exploits.
  • ctf-tools - some setup scripts for security research tools.
  • pwntools - CTF framework and exploit development library.
  • security-tools - collection of small security tools created mostly in Python. CTFs, pentests and so on.
  • pentestpackage - is a package of Pentest scripts.
  • python-pentest-tools - python tools for penetration testers.
  • fuzzdb - dictionary of attack patterns and primitives for black-box application fault injection.
  • AFL - is a free software fuzzer maintained by Google.
  • AFL++ - is AFL with community patches.
  • syzkaller - is an unsupervised, coverage-guided kernel fuzzer.
  • pwndbg - exploit development and reverse engineering with GDB made easy.
  • GDB PEDA - Python Exploit Development Assistance for GDB.
  • IDA - multi-processor disassembler and debugger useful for reverse engineering malware.
  • radare2 - framework for reverse-engineering and analyzing binaries.
  • routersploit - exploitation framework for embedded devices.
  • Ghidra - is a software reverse engineering (SRE) framework.
  • Cutter - is an SRE platform integrating Ghidra’s decompiler.
  • Vulnreport - open-source pentesting management and automation platform by Salesforce Product Security.
  • Mentalist - is a graphical tool for custom wordlist generation.
  • archerysec - vulnerability assessment and management helps to perform scans and manage vulnerabilities.
  • Osmedeus - fully automated offensive security tool for reconnaissance and vulnerability scanning.
  • beef - the browser exploitation framework project.
  • AutoSploit - automated mass exploiter.
  • SUDO\_KILLER - is a tool to identify and exploit sudo rules’ misconfigurations and vulnerabilities.
  • yara - the pattern matching swiss knife.
  • mimikatz - a little tool to play with Windows security.
  • sherlock - hunt down social media accounts by username across social networks.
  • OWASP Threat Dragon - is a tool used to create threat model diagrams and to record possible threats.

Pentests bookmarks collection

Backdoors/exploits

  • PHP-backdoors - a collection of PHP backdoors. For educational or testing purposes only.

Wordlists and Weak passwords

  • Weakpass - for any kind of bruteforce find wordlists or unleash the power of them all at once!
  • Hashes.org - is a free online hash resolving service incorporating many unparalleled techniques.
  • SecLists - collection of multiple types of lists used during security assessments, collected in one place.
  • Probable-Wordlists - sorted by probability originally created for password generation and testing.
  • skullsecurity passwords - password dictionaries and leaked passwords repository.
  • Polish PREMIUM Dictionary - official dictionary created by the team on the forum bezpieka.org. \* ^{1}
  • statistically-likely-usernames - wordlists for creating statistically likely username lists.

Bounty platforms

  • YesWeHack - bug bounty platform with infosec jobs.
  • Openbugbounty - allows any security researcher reporting a vulnerability on any website.
  • hackerone - global hacker community to surface the most relevant security issues.
  • bugcrowd - crowdsourced cybersecurity for the enterprise.
  • Crowdshield - crowdsourced security & bug bounty management.
  • Synack - crowdsourced security & bug bounty programs, crowd security intelligence platform, and more.
  • Hacktrophy - bug bounty platform.

Web Training Apps (local installation)

  • OWASP-VWAD - comprehensive and well maintained registry of all known vulnerable web applications.
  • DVWA - PHP/MySQL web application that is damn vulnerable.
  • metasploitable2 - vulnerable web application amongst security researchers.
  • metasploitable3 - is a VM that is built from the ground up with a large amount of security vulnerabilities.
  • DSVW - is a deliberately vulnerable web application written in under 100 lines of code.
  • OWASP Mutillidae II - free, open source, deliberately vulnerable web-application.
  • OWASP Juice Shop Project - the most bug-free vulnerable application in existence.
  • OWASP Node js Goat Project - OWASP Top 10 security risks apply to web apps developed using Node.js.
  • juicy-ctf - run Capture the Flags and Security Trainings with OWASP Juice Shop.
  • SecurityShepherd - web and mobile application security training platform.
  • Security Ninjas - open source application security training program.
  • hackazon - a modern vulnerable web app.
  • dvna - damn vulnerable NodeJS application.
  • django-DefectDojo - is an open-source application vulnerability correlation and security orchestration tool.
  • Google Gruyere - web application exploits and defenses.
  • Bodhi - is a playground focused on learning the exploitation of client-side web vulnerabilities.
  • Websploit - single vm lab with the purpose of combining several vulnerable appliations in one environment.
  • vulhub - pre-built Vulnerable Environments based on docker-compose.
  • CloudGoat 2 - the new & improved “Vulnerable by Design” AWS deployment tool.
  • secDevLabs - is a laboratory for learning secure web development in a practical manner.
  • CORS-vulnerable-Lab - sample vulnerable code and its exploit code.
  • RootTheBox - a Game of Hackers (CTF Scoreboard & Game Manager).
  • KONTRA - application security training (OWASP Top Web & Api).

Labs (ethical hacking platforms/trainings/CTFs)

  • Offensive Security - true performance-based penetration testing training for over a decade.
  • Hack The Box - online platform allowing you to test your penetration testing skills.
  • Hacking-Lab - online ethical hacking, computer network and security challenge platform.
  • pwnable.kr - non-commercial wargame site which provides various pwn challenges.
  • Pwnable.tw - is a wargame site for hackers to test and expand their binary exploiting skills.
  • picoCTF - is a free computer security game targeted at middle and high school students.
  • CTFlearn - is an online platform built to help ethical hackers learn and practice their cybersecurity knowledge.
  • ctftime - CTF archive and a place, where you can get some another CTF-related info.
  • Silesia Security Lab - high quality security testing services.
  • Practical Pentest Labs - pentest lab, take your Hacking skills to the next level.
  • Root Me - the fast, easy, and affordable way to train your hacking skills.
  • rozwal.to - a great platform to train your pentesting skills.
  • TryHackMe - learning Cyber Security made easy.
  • hackxor - is a realistic web application hacking game, designed to help players of all abilities develop their skills.
  • Hack Yourself First - it’s full of nasty app sec holes.
  • OverTheWire - can help you to learn and practice security concepts in the form of fun-filled games.
  • Wizard Labs - is an online Penetration Testing Lab.
  • PentesterLab - provides vulnerable systems that can be used to test and understand vulnerabilities.
  • RingZer0 - tons of challenges designed to test and improve your hacking skills.
  • try2hack - several security-oriented challenges for your entertainment.
  • Ubeeri - preconfigured lab environments.
  • Pentestit - emulate IT infrastructures of real companies for legal pen testing and improving pentest skills.
  • Microcorruption - reversal challenges done in the web interface.
  • Crackmes - download crackmes to help improve your reverse engineering skills.
  • DomGoat - DOM XSS security learning and practicing platform.
  • Stereotyped Challenges - upgrade your web hacking techniques today!
  • Vulnhub - allows anyone to gain practical ‘hands-on’ experience in digital security.
  • W3Challs - is a penetration testing training platform, which offers various computer challenges.
  • RingZer0 CTF - offers you tons of challenges designed to test and improve your hacking skills.
  • Hack.me - a platform where you can build, host and share vulnerable web apps for educational purposes.
  • HackThis! - discover how hacks, dumps and defacements are performed and secure your website.
  • Enigma Group WebApp Training - these challenges cover the exploits listed in the OWASP Top 10 Project.
  • Reverse Engineering Challenges - challenges, exercises, problems and tasks - by level, by type, and more.
  • 0x00sec - the home of the Hacker - Malware, Reverse Engineering, and Computer Science.
  • We Chall - there are exist a lots of different challenge types.
  • Hacker Gateway - is the go-to place for hackers who want to test their skills.
  • Hacker101 - is a free class for web security.
  • contained.af - a stupid game for learning about containers, capabilities, and syscalls.
  • flAWS challenge! - a series of levels you’ll learn about common mistakes and gotchas when using AWS.
  • CyberSec WTF - provides web hacking challenges derived from bounty write-ups.
  • CTF Challenge - CTF Web App challenges.
  • gCTF - most of the challenges used in the Google CTF 2017.
  • Hack This Site - is a free, safe and legal training ground for hackers.
  • Attack & Defense - is a browser-based cloud labs.
  • Cryptohack - a fun platform for learning modern cryptography.
  • Cryptopals - the cryptopals crypto challenges.

CTF platforms

  • fbctf - platform to host Capture the Flag competitions.
  • ctfscoreboard - scoreboard for Capture The Flag competitions.

Other resources

Your daily knowledge and news

RSS Readers

  • Feedly - organize, read and share what matters to you.
  • Inoreader - similar to feedly with a support for filtering what you fetch from rss.

IRC Channels

Security

  • The Hacker News - leading news source dedicated to promoting awareness for security experts and hackers.
  • Latest Hacking News - provides the latest hacking news, exploits and vulnerabilities for ethical hackers.
  • Security Newsletter - security news as a weekly digest (email notifications).
  • Google Online Security Blog - the latest news and insights from Google on security and safety on the Internet.
  • Qualys Blog - expert network security guidance and news.
  • DARKReading - connecting the Information Security Community.
  • Darknet - latest hacking tools, hacker news, cybersecurity best practices, ethical hacking & pen-testing.
  • publiclyDisclosed - public disclosure watcher who keeps you up to date about the recently disclosed bugs.
  • Reddit - Hacking - a subreddit dedicated to hacking and hackers.
  • Packet Storm - information security services, news, files, tools, exploits, advisories and whitepapers.
  • Sekurak - about security, penetration tests, vulnerabilities and many others (PL/EN).
  • nf.sec - basic aspects and mechanisms of Linux operating system security (PL).

Other/All-in-one

  • Changelog - is a community of hackers; news & podcasts for developers and hackers.

Other Cheat Sheets

Build your own DNS Servers

Build your own Certificate Authority

Build your own System/Virtual Machine

DNS Servers list (privacy)

IPURL
84.200.69.80dns.watch
94.247.43.254opennic.org
64.6.64.6verisign.com
89.233.43.71censurfridns.dk
1.1.1.1cloudflare.com
94.130.110.185dnsprivacy.at

TOP Browser extensions

Extension nameDescription
IPvFooDisplay the server IP address and HTTPS information across all page elements.
FoxyProxySimplifies configuring browsers to access proxy-servers.
HTTPS EverywhereAutomatically use HTTPS security on many sites.
uMatrixPoint & click to forbid/allow any class of requests made by your browser.
uBlock OriginAn efficient blocker: easy on memory and CPU footprint.
Session BuddyManage browser tabs and bookmarks with ease.
SuperSorterSort bookmarks recursively, delete duplicates, merge folders, and more.
Clear CacheClear your cache and browsing data.
d3coderEncoding/Decoding plugin for various types of encoding.
Web DeveloperAdds a toolbar button with various web developer tools.
ThreatPinch LookupAdd threat intelligence hover tool tips.

TOP Burp extensions

Extension nameDescription
Active Scan++Extends Burp’s active and passive scanning capabilities.
AutorizeAutomatically detects authorization enforcement.
AuthMatrixA simple matrix grid to define the desired levels of access privilege.
Logger++Logs requests and responses for all Burp tools in a sortable table.
Bypass WAFAdds headers useful for bypassing some WAF devices.
JSON BeautifierBeautifies JSON content in the HTTP message viewer.
JSON Web TokensEnables Burp to decode and manipulate JSON web tokens.
CSP AuditorDisplays CSP headers for responses, and passively reports CSP weaknesses.
CSP-BypassPassively scans for CSP headers that contain known bypasses.
HackvertorConverts data using a tag-based configuration to apply various encoding.
HTML5 AuditorScans for usage of risky HTML5 features.
Software Vulnerability ScannerVulnerability scanner based on vulners.com audit API.
Turbo IntruderIs a powerful bruteforcing tool.
Upload ScannerUpload a number of different file types, laced with different forms of payload.

Hack Mozilla Firefox address bar

In Firefox’s address bar, you can limit results by typing special characters before or after your term:

  • ^ - for matches in your browsing history
  • * - for matches in your bookmarks.
  • % - for matches in your currently open tabs.
  • # - for matches in page titles.
  • @ - for matches in web addresses.

Chrome hidden commands

  • chrome://chrome-urls - list of all commands
  • chrome://flags - enable experiments and development features
  • chrome://interstitials - errors and warnings
  • chrome://net-internals - network internals (events, dns, cache)
  • chrome://network-errors - network errors
  • chrome://net-export - start logging future network activity to a file
  • chrome://safe-browsing - safe browsing options
  • chrome://user-actions - record all user actions
  • chrome://restart - restart chrome
  • chrome://dino - ERR\_INTERNET\_DISCONNECTED…
  • cache:<website-address> - view the cached version of the web page

Bypass WAFs by Shortening IP Address (by 0xInfection)

IP addresses can be shortened by dropping the zeroes:

http://1.0.0.1 → http://1.1
http://127.0.0.1 → http://127.1
http://192.168.0.1 → http://192.168.1
 
http://0xC0A80001 or http://3232235521 → 192.168.0.1
http://192.168.257 → 192.168.1.1
http://192.168.516 → 192.168.2.4

This bypasses WAF filters for SSRF, open-redirect, etc where any IP as input gets blacklisted.

For more information please see How to Obscure Any URL and Magic IP Address Shortcuts.

Hashing, encryption and encoding (by Michal Špaček)

Hashing

plaintext ➡️ hash hash ⛔ plaintext

Symmetric encryption

plaintext ➡️ 🔑 ➡️ ciphertext plaintext ⬅️ 🔑 ⬅️ ciphertext (:key: shared key)

Asymmetric encryption

plaintext ➡️ 🔑 ➡️ ciphertext plaintext ⬅️ 〽️ ⬅️ ciphertext (:key: public key, :part\_alternation\_mark: private key)

Encoding

text ➡️ encoded text ⬅️ encoded

Shell One-liners

Table of Contents

Tool: terminal

Reload shell without exit

exec $SHELL -l

Close shell keeping all subprocess running

disown -a && exit

Exit without saving shell history

kill -9 $$
unset HISTFILE && exit

Perform a branching conditional

true && echo success
false || echo failed

Pipe stdout and stderr to separate commands

some_command > >(/bin/cmd_for_stdout) 2> >(/bin/cmd_for_stderr)

Redirect stdout and stderr each to separate files and print both to the screen

(some_command 2>&1 1>&3 | tee errorlog ) 3>&1 1>&2 | tee stdoutlog

List of commands you use most often

history | \
awk '{CMD[$2]++;count++;}END { for (a in CMD)print CMD[a] " " CMD[a]/count*100 "% " a;}' | \
grep -v "./" | \
column -c3 -s " " -t | \
sort -nr | nl |  head -n 20

Sterilize bash history

function sterile() {
 
  history | awk '$2 != "history" { $1=""; print $0 }' | egrep -vi "\
curl\b+.*(-E|--cert)\b+.*\b*|\
curl\b+.*--pass\b+.*\b*|\
curl\b+.*(-U|--proxy-user).*:.*\b*|\
curl\b+.*(-u|--user).*:.*\b*
.*(-H|--header).*(token|auth.*)\b+.*|\
wget\b+.*--.*password\b+.*\b*|\
http.?://.+:.+@.*\
" > $HOME/histbuff; history -r $HOME/histbuff;
 
}
 
export PROMPT_COMMAND="sterile"

Look also: A naive utility to censor credentials in command history.

Quickly backup a file

cp filename{,.orig}

Empty a file (truncate to 0 size)

>filename

Delete all files in a folder that don’t match a certain file extension

rm !(*.foo|*.bar|*.baz)

Pass multi-line string to a file

# cat  >filename ... - overwrite the file
# cat >>filename ... - append to a file
cat > filename << __EOF__
data data data
__EOF__

Edit a file on a remote host using vim

vim scp://user@host//etc/fstab

Create a directory and change into it at the same time

mkd() { mkdir -p "$@" && cd "$@"; }

Convert uppercase files to lowercase files

rename 'y/A-Z/a-z/' *
printf "%`tput cols`s" | tr ' ' '#'

Show shell history without line numbers

history | cut -c 8-
fc -l -n 1 | sed 's/^\s*//'

Run command(s) after exit session

cat > /etc/profile << __EOF__
_after_logout() {
 
  username=$(whoami)
 
  for _pid in $(ps afx | grep sshd | grep "$username" | awk '{print $1}') ; do
 
    kill -9 $_pid
 
  done
 
}
trap _after_logout EXIT
__EOF__

Generate a sequence of numbers

for ((i=1; i<=10; i+=2)) ; do echo $i ; done
# alternative: seq 1 2 10
 
for ((i=5; i<=10; ++i)) ; do printf '%02d\n' $i ; done
# alternative: seq -w 5 10
 
for i in {1..10} ; do echo $i ; done

Simple Bash filewatching

unset MAIL; export MAILCHECK=1; export MAILPATH='$FILE_TO_WATCH?$MESSAGE'

Tool: busybox

Static HTTP web server

busybox httpd -p $PORT -h $HOME [-c httpd.conf]

Tool: mount

Mount a temporary ram partition

mount -t tmpfs tmpfs /mnt -o size=64M
  • -t - filesystem type
  • -o - mount options

Remount a filesystem as read/write

mount -o remount,rw /

Tool: fuser

Show which processes use the files/directories

fuser /var/log/daemon.log
fuser -v /home/supervisor

Kills a process that is locking a file

fuser -ki filename
  • -i - interactive option

Kills a process that is locking a file with specific signal

fuser -k -HUP filename
  • --list-signals - list available signal names

Show what PID is listening on specific port

fuser -v 53/udp

Show all processes using the named filesystems or block device

fuser -mv /var/www

Tool: lsof

Show process that use internet connection at the moment

lsof -P -i -n

Show process that use specific port number

lsof -i tcp:443

Lists all listening ports together with the PID of the associated process

lsof -Pan -i tcp -i udp

List all open ports and their owning executables

lsof -i -P | grep -i "listen"

Show all open ports

lsof -Pnl -i

Show open ports (LISTEN)

lsof -Pni4 | grep LISTEN | column -t

List all files opened by a particular command

lsof -c "process"

View user activity per directory

lsof -u username -a +D /etc

Show 10 largest open files

lsof / | \
awk '{ if($7 > 1048576) print $7/1048576 "MB" " " $9 " " $1 }' | \
sort -n -u | tail | column -t

Show current working directory of a process

lsof -p <PID> | grep cwd

Tool: ps

Show a 4-way scrollable process tree with full details

ps awwfux | less -S

Processes per user counter

ps hax -o user | sort | uniq -c | sort -r

Show all processes by name with main header

ps -lfC nginx

Tool: find

Find files that have been modified on your system in the past 60 minutes

find / -mmin 60 -type f

Find all files larger than 20M

find / -type f -size +20M

Find duplicate files (based on MD5 hash)

find -type f -exec md5sum '{}' ';' | sort | uniq --all-repeated=separate -w 33

Change permission only for files

cd /var/www/site && find . -type f -exec chmod 766 {} \;
cd /var/www/site && find . -type f -exec chmod 664 {} +

Change permission only for directories

cd /var/www/site && find . -type d -exec chmod g+x {} \;
cd /var/www/site && find . -type d -exec chmod g+rwx {} +

Find files and directories for specific user/group

# User:
find . -user <username> -print
find /etc -type f -user <username> -name "*.conf"
 
# Group:
find /opt -group <group>
find /etc -type f -group <group> -iname "*.conf"

Find files and directories for all without specific user/group

# User:
find . \! -user <username> -print
 
# Group:
find . \! -group <group>

Looking for files/directories that only have certain permission

# User
find . -user <username> -perm -u+rw # -rw-r--r--
find /home -user $(whoami) -perm 777 # -rwxrwxrwx
 
# Group:
find /home -type d -group <group> -perm 755 # -rwxr-xr-x

Delete older files than 60 days

find . -type f -mtime +60 -delete

Recursively remove all empty sub-directories from a directory

find . -depth  -type d  -empty -exec rmdir {} \;
find </path/to/dir> -xdev -samefile filename

Recursively find the latest modified files

find . -type f -exec stat --format '%Y :%y %n' "{}" \; | sort -nr | cut -d: -f2- | head

Recursively find/replace of a string with sed

find . -not -path '*/\.git*' -type f -print0 | xargs -0 sed -i 's/foo/bar/g'

Recursively find/replace of a string in directories and file names

find . -depth -name '*test*' -execdir bash -c 'mv -v "$1" "${1//foo/bar}"' _ {} \;

Recursively find suid executables

find / \( -perm -4000 -o -perm -2000 \) -type f -exec ls -la {} \;

Tool: top

Use top to monitor only all processes with the specific string

top -p $(pgrep -d , <str>)
  • <str> - process containing string (eg. nginx, worker)

Tool: vmstat

Show current system utilization (fields in kilobytes)

vmstat 2 20 -t -w
  • 2 - number of times with a defined time interval (delay)
  • 20 - each execution of the command (count)
  • -t - show timestamp
  • -w - wide output
  • -S M - output of the fields in megabytes instead of kilobytes

Show current system utilization will get refreshed every 5 seconds

vmstat 5 -w

Display report a summary of disk operations

vmstat -D

Display report of event counters and memory stats

vmstat -s

Display report about kernel objects stored in slab layer cache

vmstat -m

Tool: iostat

Show information about the CPU usage, and I/O statistics about all the partitions

iostat 2 10 -t -m
  • 2 - number of times with a defined time interval (delay)
  • 10 - each execution of the command (count)
  • -t - show timestamp
  • -m - fields in megabytes ( -k - in kilobytes, default)

Show information only about the CPU utilization

iostat 2 10 -t -m -c

Show information only about the disk utilization

iostat 2 10 -t -m -d

Show information only about the LVM utilization

iostat -N

Tool: strace

Track with child processes

# 1)
strace -f -p $(pidof glusterfsd)
 
# 2)
strace -f $(pidof php-fpm | sed 's/\([0-9]*\)/\-p \1/g')

Track process with 30 seconds limit

timeout 30 strace $(< /var/run/zabbix/zabbix_agentd.pid)

Track processes and redirect output to a file

ps auxw | grep '[a]pache' | awk '{print " -p " $2}' | \
xargs strace -o /tmp/strace-apache-proc.out

Track with print time spent in each syscall and limit length of print strings

ps auxw | grep '[i]init_policy' | awk '{print " -p " $2}' | \
xargs strace -f -e trace=network -T -s 10000

Track the open request of a network port

strace -f -e trace=bind nc -l 80

Track the open request of a network port (show TCP/UDP)

strace -f -e trace=network nc -lu 80

Tool: kill

Kill a process running on port

kill -9 $(lsof -i :<port> | awk '{l=$2} END {print l}')

Tool: diff

Compare two directory trees

diff <(cd directory1 && find | sort) <(cd directory2 && find | sort)

Compare output of two commands

diff <(cat /etc/passwd) <(cut -f2 /etc/passwd)

Tool: vimdiff

Highlight the exact differences, based on characters and words

vimdiff file1 file2

Compare two JSON files

vimdiff <(jq -S . A.json) <(jq -S . B.json)

Compare Hex dump

d(){ vimdiff <(f $1) <(f $2);};f(){ hexdump -C $1 | cut -d' ' -f3- | tr -s ' ';}; d ~/bin1 ~/bin2

diffchar

Save diffchar @ ~/.vim/plugins

Click F7 to switch between diff modes

Usefull vimdiff commands:

  • qa to exit all windows
  • :vertical resize 70 to resize window
  • set window width Ctrl+W [N columns]+(Shift+)<\>

Tool: tail

Annotate tail -f with timestamps

tail -f file | while read ; do echo "$(date +%T.%N) $REPLY" ; done

Analyse an Apache access log for the most common IP addresses

tail -10000 access_log | awk '{print $1}' | sort | uniq -c | sort -n | tail

Analyse web server log and show only 5xx http codes

tail -n 100 -f /path/to/logfile | grep "HTTP/[1-2].[0-1]\" [5]"

Tool: tar

System backup with exclude specific directories

cd /
tar -czvpf /mnt/system$(date +%d%m%Y%s).tgz --directory=/ \
--exclude=proc/* --exclude=sys/* --exclude=dev/* --exclude=mnt/* .

System backup with exclude specific directories (pigz)

cd /
tar cvpf /backup/snapshot-$(date +%d%m%Y%s).tgz --directory=/ \
--exclude=proc/* --exclude=sys/* --exclude=dev/* \
--exclude=mnt/* --exclude=tmp/* --use-compress-program=pigz .

Tool: dump

System backup to file

dump -y -u -f /backup/system$(date +%d%m%Y%s).lzo /

Restore system from lzo file

cd /
restore -rf /backup/system$(date +%d%m%Y%s).lzo

Tool: cpulimit

Limit the cpu usage of a process

cpulimit -p pid -l 50

Tool: pwdx

Show current working directory of a process

pwdx <pid>

Tool: taskset

Start a command on only one CPU core

taskset -c 0 <command>

Tool: tr

Show directories in the PATH, one per line

tr : '\n' <<<$PATH

Tool: chmod

Remove executable bit from all files in the current directory

chmod -R -x+X *

Restore permission for /bin/chmod

# 1:
cp /bin/ls chmod.01
cp /bin/chmod chmod.01
./chmod.01 700 file
 
# 2:
/bin/busybox chmod 0700 /bin/chmod
 
# 3:
setfacl --set u::rwx,g::---,o::--- /bin/chmod

Tool: who

Find last reboot time

who -b

Detect a user sudo-su’d into the current shell

[[ $(who -m | awk '{ print $1 }') == $(whoami) ]] || echo "You are su-ed to $(whoami)"

Tool: last

Was the last reboot a panic?

(last -x -f $(ls -1t /var/log/wtmp* | head -2 | tail -1); last -x -f /var/log/wtmp) | \
grep -A1 reboot | head -2 | grep -q shutdown && echo "Expected reboot" || echo "Panic reboot"

Tool: screen

Start screen in detached mode

screen -d -m <command>

Attach to an existing screen session

screen -r -d <pid>

Tool: script

Record and replay terminal session

### Record session
# 1)
script -t 2>~/session.time -a ~/session.log
 
# 2)
script --timing=session.time session.log
 
### Replay session
scriptreplay --timing=session.time session.log

Tool: du

Show 20 biggest directories with ‘K M G’

du | \
sort -r -n | \
awk '{split("K M G",v); s=1; while($1>1024){$1/=1024; s++} print int($1)" "v[s]"\t"$2}' | \
head -n 20

Tool: inotifywait

Init tool everytime a file in a directory is modified

while true ; do inotifywait -r -e MODIFY dir/ && ls dir/ ; done;

Tool: openssl

Testing connection to the remote host

echo | openssl s_client -connect google.com:443 -showcerts

Testing connection to the remote host (debug mode)

echo | openssl s_client -connect google.com:443 -showcerts -tlsextdebug -status

Testing connection to the remote host (with SNI support)

echo | openssl s_client -showcerts -servername google.com -connect google.com:443

Testing connection to the remote host with specific ssl version

openssl s_client -tls1_2 -connect google.com:443

Testing connection to the remote host with specific ssl cipher

openssl s_client -cipher 'AES128-SHA' -connect google.com:443

Verify 0-RTT

_host="example.com"
 
cat > req.in << __EOF__
HEAD / HTTP/1.1
Host: $_host
Connection: close
__EOF__
 
openssl s_client -connect ${_host}:443 -tls1_3 -sess_out session.pem -ign_eof < req.in
openssl s_client -connect ${_host}:443 -tls1_3 -sess_in session.pem -early_data req.in

Generate private key without passphrase

# _len: 2048, 4096
( _fd="private.key" ; _len="2048" ; \
openssl genrsa -out ${_fd} ${_len} )

Generate private key with passphrase

# _ciph: aes128, aes256
# _len: 2048, 4096
( _ciph="aes128" ; _fd="private.key" ; _len="2048" ; \
openssl genrsa -${_ciph} -out ${_fd} ${_len} )

Remove passphrase from private key

( _fd="private.key" ; _fd_unp="private_unp.key" ; \
openssl rsa -in ${_fd} -out ${_fd_unp} )

Encrypt existing private key with a passphrase

# _ciph: aes128, aes256
( _ciph="aes128" ; _fd="private.key" ; _fd_pass="private_pass.key" ; \
openssl rsa -${_ciph} -in ${_fd} -out ${_fd_pass}

Check private key

( _fd="private.key" ; \
openssl rsa -check -in ${_fd} )

Get public key from private key

( _fd="private.key" ; _fd_pub="public.key" ; \
openssl rsa -pubout -in ${_fd} -out ${_fd_pub} )

Generate private key and CSR

( _fd="private.key" ; _fd_csr="request.csr" ; _len="2048" ; \
openssl req -out ${_fd_csr} -new -newkey rsa:${_len} -nodes -keyout ${_fd} )

Generate CSR

( _fd="private.key" ; _fd_csr="request.csr" ; \
openssl req -out ${_fd_csr} -new -key ${_fd} )

Generate CSR (metadata from existing certificate)

Where private.key is the existing private key. As you can see you do not generate this CSR from your certificate (public key). Also you do not generate the “same” CSR, just a new one to request a new certificate.

( _fd="private.key" ; _fd_csr="request.csr" ; _fd_crt="cert.crt" ; \
openssl x509 -x509toreq -in ${_fd_crt} -out ${_fd_csr} -signkey ${_fd} )

Generate CSR with -config param

( _fd="private.key" ; _fd_csr="request.csr" ; \
openssl req -new -sha256 -key ${_fd} -out ${_fd_csr} \
-config <(
cat << __EOF__
[req]
default_bits        = 2048
default_md          = sha256
prompt              = no
distinguished_name  = dn
req_extensions      = req_ext
 
[ dn ]
C   = "<two-letter ISO abbreviation for your country>"
ST  = "<state or province where your organisation is legally located>"
L   = "<city where your organisation is legally located>"
O   = "<legal name of your organisation>"
OU  = "<section of the organisation>"
CN  = "<fully qualified domain name>"
 
[ req_ext ]
subjectAltName = @alt_names
 
[ alt_names ]
DNS.1 = <fully qualified domain name>
DNS.2 = <next domain>
DNS.3 = <next domain>
__EOF__
))

Other values in [ dn ] :

countryName            = "DE"                     # C=
stateOrProvinceName    = "Hessen"                 # ST=
localityName           = "Keller"                 # L=
postalCode             = "424242"                 # L/postalcode=
postalAddress          = "Keller"                 # L/postaladdress=
streetAddress          = "Crater 1621"            # L/street=
organizationName       = "apfelboymschule"        # O=
organizationalUnitName = "IT Department"          # OU=
commonName             = "example.com"            # CN=
emailAddress           = "webmaster@example.com"  # CN/emailAddress=

Example of oids (you’ll probably also have to make OpenSSL know about the new fields required for EV by adding the following under [new_oids] ):

[req]
...
oid_section         = new_oids
 
[ new_oids ]
postalCode = 2.5.4.17
streetAddress = 2.5.4.9

Full example:

( _fd="private.key" ; _fd_csr="request.csr" ; \
openssl req -new -sha256 -key ${_fd} -out ${_fd_csr} \
-config <(
cat << __EOF__
[req]
default_bits        = 2048
default_md          = sha256
prompt              = no
distinguished_name  = dn
req_extensions      = req_ext
oid_section         = new_oids
 
[ new_oids ]
serialNumber = 2.5.4.5
streetAddress = 2.5.4.9
postalCode = 2.5.4.17
businessCategory = 2.5.4.15
 
[ dn ]
serialNumber=00001111
businessCategory=Private Organization
jurisdictionC=DE
C=DE
ST=Hessen
L=Keller
postalCode=424242
streetAddress=Crater 1621
O=AV Company
OU=IT
CN=example.com
 
[ req_ext ]
subjectAltName = @alt_names
 
[ alt_names ]
DNS.1 = example.com
__EOF__
))

For more information please look at these great explanations:

List available EC curves

openssl ecparam -list_curves
( _fd="private.key" ; \
openssl ec -in ${_fd} -noout -text )
 
# For x25519 only extracting public key
( _fd="private.key" ; _fd_pub="public.key" ; \
openssl pkey -in ${_fd} -pubout -out ${_fd_pub} )

Generate ECDSA private key

# _curve: prime256v1, secp521r1, secp384r1
( _fd="private.key" ; _curve="prime256v1" ; \
openssl ecparam -out ${_fd} -name ${_curve} -genkey )
 
# _curve: X25519
( _fd="private.key" ; _curve="x25519" ; \
openssl genpkey -algorithm ${_curve} -out ${_fd} )

Generate private key and CSR (ECC)

# _curve: prime256v1, secp521r1, secp384r1
( _fd="domain.com.key" ; _fd_csr="domain.com.csr" ; _curve="prime256v1" ; \
openssl ecparam -out ${_fd} -name ${_curve} -genkey ; \
openssl req -new -key ${_fd} -out ${_fd_csr} -sha256 )

Generate self-signed certificate

# _len: 2048, 4096
( _fd="domain.key" ; _fd_out="domain.crt" ; _len="2048" ; _days="365" ; \
openssl req -newkey rsa:${_len} -nodes \
-keyout ${_fd} -x509 -days ${_days} -out ${_fd_out} )

Generate self-signed certificate from existing private key

# _len: 2048, 4096
( _fd="domain.key" ; _fd_out="domain.crt" ; _days="365" ; \
openssl req -key ${_fd} -nodes \
-x509 -days ${_days} -out ${_fd_out} )

Generate self-signed certificate from existing private key and csr

# _len: 2048, 4096
( _fd="domain.key" ; _fd_csr="domain.csr" ; _fd_out="domain.crt" ; _days="365" ; \
openssl x509 -signkey ${_fd} -nodes \
-in ${_fd_csr} -req -days ${_days} -out ${_fd_out} )

Generate DH public parameters

( _dh_size="2048" ; \
openssl dhparam -out /etc/nginx/ssl/dhparam_${_dh_size}.pem "$_dh_size" )

Display DH public parameters

openssl pkeyparam -in dhparam.pem -text

Extract private key from pfx

( _fd_pfx="cert.pfx" ; _fd_key="key.pem" ; \
openssl pkcs12 -in ${_fd_pfx} -nocerts -nodes -out ${_fd_key} )

Extract private key and certs from pfx

( _fd_pfx="cert.pfx" ; _fd_pem="key_certs.pem" ; \
openssl pkcs12 -in ${_fd_pfx} -nodes -out ${_fd_pem} )

Extract certs from p7b

# PKCS#7 file doesn't include private keys.
( _fd_p7b="cert.p7b" ; _fd_pem="cert.pem" ; \
openssl pkcs7 -inform DER -outform PEM -in ${_fd_p7b} -print_certs > ${_fd_pem})
# or:
openssl pkcs7 -print_certs -in -in ${_fd_p7b} -out ${_fd_pem})

Convert DER to PEM

( _fd_der="cert.crt" ; _fd_pem="cert.pem" ; \
openssl x509 -in ${_fd_der} -inform der -outform pem -out ${_fd_pem} )

Convert PEM to DER

( _fd_der="cert.crt" ; _fd_pem="cert.pem" ; \
openssl x509 -in ${_fd_pem} -outform der -out ${_fd_der} )

Verification of the private key

( _fd="private.key" ; \
openssl rsa -noout -text -in ${_fd} )

Verification of the public key

# 1)
( _fd="public.key" ; \
openssl pkey -noout -text -pubin -in ${_fd} )
 
# 2)
( _fd="private.key" ; \
openssl rsa -inform PEM -noout -in ${_fd} &> /dev/null ; \
if [ $? = 0 ] ; then echo -en "OK\n" ; fi )

Verification of the certificate

( _fd="certificate.crt" ; # format: pem, cer, crt \
openssl x509 -noout -text -in ${_fd} )

Verification of the CSR

( _fd_csr="request.csr" ; \
openssl req -text -noout -in ${_fd_csr} )

Check the private key and the certificate are match

(openssl rsa -noout -modulus -in private.key | openssl md5 ; \
openssl x509 -noout -modulus -in certificate.crt | openssl md5) | uniq

Check the private key and the CSR are match

(openssl rsa -noout -modulus -in private.key | openssl md5 ; \
openssl req -noout -modulus -in request.csr | openssl md5) | uniq

Tool: secure-delete

Secure delete with shred

shred -vfuz -n 10 file
shred --verbose --random-source=/dev/urandom -n 1 /dev/sda

Secure delete with scrub

scrub -p dod /dev/sda
scrub -p dod -r file

Secure delete with badblocks

badblocks -s -w -t random -v /dev/sda
badblocks -c 10240 -s -w -t random -v /dev/sda

Secure delete with secure-delete

srm -vz /tmp/file
sfill -vz /local
sdmem -v
swapoff /dev/sda5 && sswap -vz /dev/sda5

Tool: dd

Show dd status every so often

dd <dd_params> status=progress
watch --interval 5 killall -USR1 dd

Redirect output to a file with dd

echo "string" | dd of=filename

Tool: gpg

Export public key

gpg --export --armor "<username>" > username.pkey
  • --export - export all keys from all keyrings or specific key
  • -a|--armor - create ASCII armored output

Encrypt file

gpg -e -r "<username>" dump.sql
  • -e|--encrypt - encrypt data
  • -r|--recipient - encrypt for specific

Decrypt file

gpg -o dump.sql -d dump.sql.gpg
  • -o|--output - use as output file
  • -d|--decrypt - decrypt data (default)

Search recipient

gpg --keyserver hkp://keyserver.ubuntu.com --search-keys "<username>"
  • --keyserver - set specific key server
  • --search-keys - search for keys on a key server

List all of the packets in an encrypted file

gpg --batch --list-packets archive.gpg
gpg2 --batch --list-packets archive.gpg

Tool: system-other

Reboot system from init

exec /sbin/init 6

Init system from single user mode

exec /sbin/init

Show current working directory of a process

readlink -f /proc/<PID>/cwd

Show actual pathname of the executed command

readlink -f /proc/<PID>/exe

Tool: curl

curl -Iks https://www.google.com
  • -I - show response headers only
  • -k - insecure connection when using ssl
  • -s - silent mode (not display body)
curl -Iks --location -X GET -A "x-agent" https://www.google.com
  • --location - follow redirects
  • -X - set method
  • -A - set user-agent
curl -Iks --location -X GET -A "x-agent" --proxy http://127.0.0.1:16379 https://www.google.com
  • --proxy [socks5://|http://] - set proxy server
curl -o file.pdf -C - https://example.com/Aiju2goo0Ja2.pdf
  • -o - write output to file
  • -C - resume the transfer

Find your external IP address (external services)

curl ipinfo.io
curl ipinfo.io/ip
curl icanhazip.com
curl ifconfig.me/ip ; echo

Repeat URL request

# URL sequence substitution with a dummy query string:
curl -ks https://example.com/?[1-20]
 
# With shell 'for' loop:
for i in {1..20} ; do curl -ks https://example.com/ ; done

Check DNS and HTTP trace with headers for specific domains

### Set domains and external dns servers.
_domain_list=(google.com) ; _dns_list=("8.8.8.8" "1.1.1.1")
 
for _domain in "${_domain_list[@]}" ; do
 
  printf '=%.0s' {1..48}
 
  echo
 
  printf "[\\e[1;32m+\\e[m] resolve: %s\\n" "$_domain"
 
  for _dns in "${_dns_list[@]}" ; do
 
    # Resolve domain.
    host "${_domain}" "${_dns}"
 
    echo
 
  done
 
  for _proto in http https ; do
 
    printf "[\\e[1;32m+\\e[m] trace + headers: %s://%s\\n" "$_proto" "$_domain"
 
    # Get trace and http headers.
    curl -Iks -A "x-agent" --location "${_proto}://${_domain}"
 
    echo
 
  done
 
done
 
unset _domain_list _dns_list

Tool: httpie

http -p Hh https://www.google.com
  • -p - print request and response headers
    • H - request headers
    • B - request body
    • h - response headers
    • b - response body
http -p Hh https://www.google.com --follow --verify no
  • -F, --follow - follow redirects
  • --verify no - skip SSL verification
http -p Hh https://www.google.com --follow --verify no \
--proxy http:http://127.0.0.1:16379
  • --proxy [http:] - set proxy server

Tool: ssh

Escape Sequence

# Supported escape sequences:
~.  - terminate connection (and any multiplexed sessions)
~B  - send a BREAK to the remote system
~C  - open a command line
~R  - Request rekey (SSH protocol 2 only)
~^Z - suspend ssh
~#  - list forwarded connections
~&  - background ssh (when waiting for connections to terminate)
~?  - this message
~~  - send the escape character by typing it twice

Compare a remote file with a local file

ssh user@host cat /path/to/remotefile | diff /path/to/localfile -

SSH connection through host in the middle

ssh -t reachable_host ssh unreachable_host

Run command over SSH on remote host

cat > cmd.txt << __EOF__
cat /etc/hosts
__EOF__
 
ssh host -l user $(<cmd.txt)

Get public key from private key

ssh-keygen -y -f ~/.ssh/id_rsa

Get all fingerprints

ssh-keygen -l -f .ssh/known_hosts

SSH authentication with user password

ssh -o PreferredAuthentications=password -o PubkeyAuthentication=no user@remote_host

SSH authentication with publickey

ssh -o PreferredAuthentications=publickey -o PubkeyAuthentication=yes -i id_rsa user@remote_host

Simple recording SSH session

function _ssh_sesslog() {
 
  _sesdir="<path/to/session/logs>"
 
  mkdir -p "${_sesdir}" && \
  ssh $@ 2>&1 | tee -a "${_sesdir}/$(date +%Y%m%d).log"
 
}
 
# Alias:
alias ssh='_ssh_sesslog'

Using Keychain for SSH logins

### Delete all of ssh-agent's keys.
function _scl() {
 
  /usr/bin/keychain --clear
 
}
 
### Add key to keychain.
function _scg() {
 
  /usr/bin/keychain /path/to/private-key
  source "$HOME/.keychain/$HOSTNAME-sh"
 
}

SSH login without processing any login scripts

ssh -tt user@host bash

SSH local port forwarding

Example 1:

# Forwarding our local 2250 port to nmap.org:443 from localhost through localhost
host1> ssh -L 2250:nmap.org:443 localhost
 
# Connect to the service:
host1> curl -Iks --location -X GET https://localhost:2250

Example 2:

# Forwarding our local 9051 port to db.d.x:5432 from localhost through node.d.y
host1> ssh -nNT -L 9051:db.d.x:5432 node.d.y
 
# Connect to the service:
host1> psql -U db_user -d db_dev -p 9051 -h localhost
  • -n - redirects stdin from /dev/null
  • -N - do not execute a remote command
  • -T - disable pseudo-terminal allocation

SSH remote port forwarding

# Forwarding our local 9051 port to db.d.x:5432 from host2 through node.d.y
host1> ssh -nNT -R 9051:db.d.x:5432 node.d.y
 
# Connect to the service:
host2> psql -U postgres -d postgres -p 8000 -h localhost

Tool: linux-dev

Testing remote connection to port

timeout 1 bash -c "</dev/<proto>/<host>/<port>" >/dev/null 2>&1 ; echo $?
  • <proto - set protocol (tcp/udp)
  • <host> - set remote host
  • <port> - set destination port

Read and write to TCP or UDP sockets with common bash tools

exec 5<>/dev/tcp/<host>/<port>; cat <&5 & cat >&5; exec 5>&-

Tool: tcpdump

Filter incoming (on interface) traffic (specific ip:port)

tcpdump -ne -i eth0 -Q in host 192.168.252.1 and port 443
  • -n - don’t convert addresses ( -nn will not resolve hostnames or ports)
  • -e - print the link-level headers
  • -i [iface|any] - set interface
  • -Q|-D [in|out|inout] - choose send/receive direction ( -D - for old tcpdump versions)
  • host [ip|hostname] - set host, also [host not]
  • [and|or] - set logic
  • port [1-65535] - set port number, also [port not]

Filter incoming (on interface) traffic (specific ip:port) and write to a file

tcpdump -ne -i eth0 -Q in host 192.168.252.1 and port 443 -c 5 -w tcpdump.pcap
  • -c [num] - capture only num number of packets
  • -w [filename] - write packets to file, -r [filename] - reading from file

Capture all ICMP packets

tcpdump -nei eth0 icmp

Check protocol used (TCP or UDP) for service

tcpdump -nei eth0 tcp port 22 -vv -X | egrep "TCP|UDP"

Display ASCII text (to parse the output using grep or other)

tcpdump -i eth0 -A -s0 port 443

Grab everything between two keywords

tcpdump -i eth0 port 80 -X | sed -n -e '/username/,/=ldap/ p'

Grab user and pass ever plain http

tcpdump -i eth0  port http -l -A | egrep -i \
'pass=|pwd=|log=|login=|user=|username=|pw=|passw=|passwd=|password=|pass:|user:|username:|password:|login:|pass |user ' \
--color=auto --line-buffered -B20

Extract HTTP User Agent from HTTP request header

tcpdump -ei eth0 -nn -A -s1500 -l | grep "User-Agent:"

Capture only HTTP GET and POST packets

tcpdump -ei eth0 -s 0 -A -vv \
'tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x47455420' or 'tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x504f5354'

or simply:

tcpdump -ei eth0 -s 0 -v -n -l | egrep -i "POST /|GET /|Host:"

Rotate capture files

tcpdump -ei eth0 -w /tmp/capture-%H.pcap -G 3600 -C 200
  • -G <num> - pcap will be created every <num> seconds
  • -C <size> - close the current pcap and open a new one if is larger than <size>

Top hosts by packets

tcpdump -ei enp0s25 -nnn -t -c 200 | cut -f 1,2,3,4 -d '.' | sort | uniq -c | sort -nr | head -n 20

Excludes any RFC 1918 private address

tcpdump -nei eth0 'not (src net (10 or 172.16/12 or 192.168/16) and dst net (10 or 172.16/12 or 192.168/16))'

Tool: tcpick

Analyse packets in real-time

while true ; do tcpick -a -C -r dump.pcap ; sleep 2 ; clear ; done

Tool: ngrep

ngrep -d eth0 "www.domain.com" port 443
  • -d [iface|any] - set interface
  • [domain] - set hostname
  • port [1-65535] - set port number
ngrep -d eth0 "www.domain.com" src host 10.240.20.2 and port 443
  • (host [ip|hostname]) - filter by ip or hostname
  • (port [1-65535]) - filter by port number
ngrep -d eth0 -qt -O ngrep.pcap "www.domain.com" port 443
  • -q - quiet mode (only payloads)
  • -t - added timestamps
  • -O [filename] - save output to file, -I [filename] - reading from file
ngrep -d eth0 -qt 'HTTP' 'tcp'
  • HTTP - show http headers
  • tcp|udp - set protocol
  • [src|dst] host [ip|hostname] - set direction for specific node
ngrep -l -q -d eth0 -i "User-Agent: curl*"
  • -l - stdout line buffered
  • -i - case-insensitive search

Tool: hping3

hping3 -V -p 80 -s 5050 <scan_type> www.google.com
  • -V|--verbose - verbose mode
  • -p|--destport - set destination port
  • -s|--baseport - set source port
  • <scan_type> - set scan type
    • -F|--fin - set FIN flag, port open if no reply
    • -S|--syn - set SYN flag
    • -P|--push - set PUSH flag
    • -A|--ack - set ACK flag (use when ping is blocked, RST response back if the port is open)
    • -U|--urg - set URG flag
    • -Y|--ymas - set Y unused flag (0x80 - nullscan), port open if no reply
    • -M 0 -UPF - set TCP sequence number and scan type (URG+PUSH+FIN), port open if no reply
hping3 -V -c 1 -1 -C 8 www.google.com
  • -c [num] - packet count
  • -1 - set ICMP mode
  • -C|--icmptype [icmp-num] - set icmp type (default icmp-echo = 8)
hping3 -V -c 1000000 -d 120 -S -w 64 -p 80 --flood --rand-source <remote_host>
  • --flood - sent packets as fast as possible (don’t show replies)
  • --rand-source - random source address mode
  • -d --data - data size
  • -w|--win - winsize (default 64)

Tool: nmap

Ping scans the network

nmap -sP 192.168.0.0/24

Show only open ports

nmap -F --open 192.168.0.0/24

Full TCP port scan using with service version detection

nmap -p 1-65535 -sV -sS -T4 192.168.0.0/24

Nmap scan and pass output to Nikto

nmap -p80,443 192.168.0.0/24 -oG - | nikto.pl -h -

Recon specific ip:service with Nmap NSE scripts stack

# Set variables:
_hosts="192.168.250.10"
_ports="80,443"
 
# Set Nmap NSE scripts stack:
_nmap_nse_scripts="+dns-brute,\
                   +http-auth-finder,\
                   +http-chrono,\
                   +http-cookie-flags,\
                   +http-cors,\
                   +http-cross-domain-policy,\
                   +http-csrf,\
                   +http-dombased-xss,\
                   +http-enum,\
                   +http-errors,\
                   +http-git,\
                   +http-grep,\
                   +http-internal-ip-disclosure,\
                   +http-jsonp-detection,\
                   +http-malware-host,\
                   +http-methods,\
                   +http-passwd,\
                   +http-phpself-xss,\
                   +http-php-version,\
                   +http-robots.txt,\
                   +http-sitemap-generator,\
                   +http-shellshock,\
                   +http-stored-xss,\
                   +http-title,\
                   +http-unsafe-output-escaping,\
                   +http-useragent-tester,\
                   +http-vhosts,\
                   +http-waf-detect,\
                   +http-waf-fingerprint,\
                   +http-xssed,\
                   +traceroute-geolocation.nse,\
                   +ssl-enum-ciphers,\
                   +whois-domain,\
                   +whois-ip"
 
# Set Nmap NSE script params:
_nmap_nse_scripts_args="dns-brute.domain=${_hosts},http-cross-domain-policy.domain-lookup=true,"
_nmap_nse_scripts_args+="http-waf-detect.aggro,http-waf-detect.detectBodyChanges,"
_nmap_nse_scripts_args+="http-waf-fingerprint.intensive=1"
 
# Perform scan:
nmap --script="$_nmap_nse_scripts" --script-args="$_nmap_nse_scripts_args" -p "$_ports" "$_hosts"

Tool: netcat

nc -kl 5000
  • -l - listen for an incoming connection
  • -k - listening after client has disconnected
  • >filename.out - save receive data to file (optional)
nc 192.168.0.1 5051 < filename.in
  • < filename.in - send data to remote host
nc -vz 10.240.30.3 5000
  • -v - verbose output
  • -z - scan for listening daemons
nc -vzu 10.240.30.3 1-65535
  • -u - scan only udp ports

Transfer data file (archive)

server> nc -l 5000 | tar xzvfp -
client> tar czvfp - /path/to/dir | nc 10.240.30.3 5000

Launch remote shell

# 1)
server> nc -l 5000 -e /bin/bash
client> nc 10.240.30.3 5000
 
# 2)
server> rm -f /tmp/f; mkfifo /tmp/f
server> cat /tmp/f | /bin/bash -i 2>&1 | nc -l 127.0.0.1 5000 > /tmp/f
client> nc 10.240.30.3 5000

Simple file server

while true ; do nc -l 5000 | tar -xvf - ; done

Simple minimal HTTP Server

while true ; do nc -l -p 1500 -c 'echo -e "HTTP/1.1 200 OK\n\n $(date)"' ; done

Simple HTTP Server

Restarts web server after each request - remove while condition for only single connection.

cat > index.html << __EOF__
<!doctype html>
    <head>
        <meta charset="utf-8">
        <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
        <title></title>
        <meta name="description" content="">
        <meta name="viewport" content="width=device-width, initial-scale=1">
    </head>
    <body>
 
    <p>
 
      Hello! It's a site.
 
    </p>
 
    </body>
</html>
__EOF__
server> while : ; do \
(echo -ne "HTTP/1.1 200 OK\r\nContent-Length: $(wc -c <index.html)\r\n\r\n" ; cat index.html;) | \
nc -l -p 5000 \
; done
  • -p - port number

Simple HTTP Proxy (single connection)

#!/usr/bin/env bash
 
if [[ $#--2-| != 2 ]] ; then
  printf "%s\\n" \
         "usage: ./nc-proxy listen-port bk_host:bk_port"
fi
 
_listen_port="$1"
_bk_host=$(echo "$2" | cut -d ":" -f1)
_bk_port=$(echo "$2" | cut -d ":" -f2)
 
printf "  lport: %s\\nbk_host: %s\\nbk_port: %s\\n\\n" \
       "$_listen_port" "$_bk_host" "$_bk_port"
 
_tmp=$(mktemp -d)
_back="$_tmp/pipe.back"
_sent="$_tmp/pipe.sent"
_recv="$_tmp/pipe.recv"
 
trap 'rm -rf "$_tmp"' EXIT
 
mkfifo -m 0600 "$_back" "$_sent" "$_recv"
 
sed "s/^/=> /" <"$_sent" &
sed "s/^/<=  /" <"$_recv" &
 
nc -l -p "$_listen_port" <"$_back" | \
tee "$_sent" | \
nc "$_bk_host" "$_bk_port" | \
tee "$_recv" >"$_back"
server> chmod +x nc-proxy && ./nc-proxy 8080 192.168.252.10:8000
  lport: 8080
bk_host: 192.168.252.10
bk_port: 8000
 
client> http -p h 10.240.30.3:8080
HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: max-age=31536000
Content-Length: 2748
Content-Type: text/html; charset=utf-8
Date: Sun, 01 Jul 2018 20:12:08 GMT
Last-Modified: Sun, 01 Apr 2018 21:53:37 GMT

Create a single-use TCP or UDP proxy

### TCP -> TCP
nc -l -p 2000 -c "nc [ip|hostname] 3000"
 
### TCP -> UDP
nc -l -p 2000 -c "nc -u [ip|hostname] 3000"
 
### UDP -> UDP
nc -l -u -p 2000 -c "nc -u [ip|hostname] 3000"
 
### UDP -> TCP
nc -l -u -p 2000 -c "nc [ip|hostname] 3000"

Tool: gnutls-cli

Testing connection to remote host (with SNI support)

gnutls-cli -p 443 google.com

Testing connection to remote host (without SNI support)

gnutls-cli --disable-sni -p 443 google.com

Tool: socat

Testing remote connection to port

socat - TCP4:10.240.30.3:22
  • - - standard input (STDIO)
  • TCP4:<params> - set tcp4 connection with specific params
    • [hostname|ip] - set hostname/ip
    • [1-65535] - set port number

Redirecting TCP-traffic to a UNIX domain socket under Linux

socat TCP-LISTEN:1234,bind=127.0.0.1,reuseaddr,fork,su=nobody,range=127.0.0.0/8 UNIX-CLIENT:/tmp/foo
  • TCP-LISTEN:<params> - set tcp listen with specific params
    • [1-65535] - set port number
    • bind=[hostname|ip] - set bind hostname/ip
    • reuseaddr - allows other sockets to bind to an address
    • fork - keeps the parent process attempting to produce more connections
    • su=nobody - set user
    • range=[ip-range] - ip range
  • UNIX-CLIENT:<params> - communicates with the specified peer socket
    • filename - define socket

Tool: p0f

Set iface in promiscuous mode and dump traffic to the log file

p0f -i enp0s25 -p -d -o /dump/enp0s25.log
  • -i - listen on the specified interface
  • -p - set interface in promiscuous mode
  • -d - fork into background
  • -o - output file

Tool: netstat

Graph # of connections for each hosts

netstat -an | awk '/ESTABLISHED/ { split($5,ip,":"); if (ip[1] !~ /^$/) print ip[1] }' | \
sort | uniq -c | awk '{ printf("%s\t%s\t",$2,$1) ; for (i = 0; i < $1; i++) {printf("*")}; print "" }'

Monitor open connections for specific port including listen, count and sort it per IP

watch "netstat -plan | grep :443 | awk {'print \$5'} | cut -d: -f 1 | sort | uniq -c | sort -nk 1"

Grab banners from local IPv4 listening ports

netstat -nlt | grep 'tcp ' | grep -Eo "[1-9][0-9]*" | xargs -I {} sh -c "echo "" | nc -v -n -w1 127.0.0.1 {}"

Tool: rsync

Rsync remote data as root using sudo

rsync --rsync-path 'sudo rsync' username@hostname:/path/to/dir/ /local/

Tool: host

Resolves the domain name (using external dns server)

host google.com 9.9.9.9

Checks the domain administrator (SOA record)

host -t soa google.com 9.9.9.9

Tool: dig

Resolves the domain name (short output)

dig google.com +short

Lookup NS record for specific domain

dig @9.9.9.9 google.com NS

Query only answer section

dig google.com +nocomments +noquestion +noauthority +noadditional +nostats

Query ALL DNS Records

dig google.com ANY +noall +answer

DNS Reverse Look-up

dig -x 172.217.16.14 +short

Tool: certbot

Generate multidomain certificate

certbot certonly -d example.com -d www.example.com

Generate wildcard certificate

certbot certonly --manual --preferred-challenges=dns -d example.com -d *.example.com

Generate certificate with 4096 bit private key

certbot certonly -d example.com -d www.example.com --rsa-key-size 4096

Tool: network-other

Get all subnets for specific AS (Autonomous system)

AS="AS32934"
whois -h whois.radb.net -- "-i origin ${AS}" | \
grep "^route:" | \
cut -d ":" -f2 | \
sed -e 's/^[ \t]//' | \
sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4 | \
cut -d ":" -f2 | \
sed -e 's/^[ \t]/allow /' | \
sed 's/$/;/' | \
sed 's/allow  */subnet -> /g'

Resolves domain name from dns.google.com with curl and jq

_dname="google.com" ; curl -s "https://dns.google.com/resolve?name=${_dname}&type=A" | jq .

Tool: git

Log alias for a decent view of your repo

# 1)
git log --oneline --decorate --graph --all
 
# 2)
git log --graph \
--pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an>%Creset' \
--abbrev-commit

Tool: python

Static HTTP web server

# Python 3.x
python3 -m http.server 8000 --bind 127.0.0.1
 
# Python 2.x
python -m SimpleHTTPServer 8000

Static HTTP web server with SSL support

# Python 3.x
from http.server import HTTPServer, BaseHTTPRequestHandler
import ssl
 
httpd = HTTPServer(('localhost', 4443), BaseHTTPRequestHandler)
 
httpd.socket = ssl.wrap_socket (httpd.socket,
        keyfile="path/to/key.pem",
        certfile='path/to/cert.pem', server_side=True)
 
httpd.serve_forever()
 
# Python 2.x
import BaseHTTPServer, SimpleHTTPServer
import ssl
 
httpd = BaseHTTPServer.HTTPServer(('localhost', 4443),
        SimpleHTTPServer.SimpleHTTPRequestHandler)
 
httpd.socket = ssl.wrap_socket (httpd.socket,
        keyfile="path/tp/key.pem",
        certfile='path/to/cert.pem', server_side=True)
 
httpd.serve_forever()

Encode base64

python -m base64 -e <<< "sample string"

Decode base64

python -m base64 -d <<< "dGhpcyBpcyBlbmNvZGVkCg=="

Tool: awk

Search for matching lines

# egrep foo
awk '/foo/' filename

Search non matching lines

# egrep -v foo
awk '!/foo/' filename
# egrep -n foo
awk '/foo/{print FNR,$0}' filename
awk '{print $NF}' filename

Find all the lines longer than 80 characters

awk 'length($0)>80{print FNR,$0}' filename
awk 'length < 80' filename
awk '1; { print "" }' filename
awk '{ print FNR "\t" $0 }' filename
awk '{ printf("%5d : %s\n", NR, $0) }' filename   # in a fancy manner
awk 'NF { $0=++a " :" $0 }; { print }' filename
awk '/foo/{i=5+1;}{if(i){i--; print;}}' filename

Print the lines starting at the line matching ‘server {’ until the line matching ’}‘

awk '/server {/,/}/' filename
awk -F' ' '{print "ip:\t" $2 "\n port:\t" $3' filename

Remove empty lines

awk 'NF > 0' filename
 
# alternative:
awk NF filename

Delete trailing white space (spaces, tabs)

awk '{sub(/[ \t]*$/, "");print}' filename

Delete leading white space

awk '{sub(/^[ \t]+/, ""); print}' filename

Remove duplicate consecutive lines

# uniq
awk 'a !~ $0{print}; {a=$0}' filename

Remove duplicate entries in a file without sorting

awk '!x[$0]++' filename

Exclude multiple columns

awk '{$1=$3=""}1' filename

Substitute foo for bar on lines matching regexp

awk '/regexp/{gsub(/foo/, "bar")};{print}' filename

Add some characters at the beginning of matching lines

awk '/regexp/{sub(/^/, "++++"); print;next;}{print}' filename

Get the last hour of Apache logs

awk '/'$(date -d "1 hours ago" "+%d\\/%b\\/%Y:%H:%M")'/,/'$(date "+%d\\/%b\\/%Y:%H:%M")'/ { print $0 }' \
/var/log/httpd/access_log

Tool: sed

sed -n 10p /path/to/file

Remove a specific line from a file

sed -i 10d /path/to/file
# alternative (BSD): sed -i'' 10d /path/to/file

Remove a range of lines from a file

sed -i <file> -re '<start>,<end>d'

Replace newline(s) with a space

sed ':a;N;$!ba;s/\n/ /g' /path/to/file
 
# cross-platform compatible syntax:
sed -e ':a' -e 'N' -e '$!ba' -e 's/\n/ /g' /path/to/file
  • :a create a label a
  • N append the next line to the pattern space
  • $! if not the last line, ba branch (go to) label a
  • s substitute, /\n/ regex for new line, / / by a space, /g global match (as many times as it can)

Alternatives:

# perl version (sed-like speed):
perl -p -e 's/\n/ /' /path/to/file
 
# bash version (slow):
while read line ; do printf "%s" "$line " ; done < file

Delete string +N next lines

sed '/start/,+4d' /path/to/file

Tool: grep

Search for a “pattern” inside all files in the current directory

grep -rn "pattern"
grep -RnisI "pattern" *
fgrep "pattern" * -R

Show only for multiple patterns

grep 'INFO*'\''WARN' filename
grep 'INFO\|WARN' filename
grep -e INFO -e WARN filename
grep -E '(INFO|WARN)' filename
egrep "INFO|WARN" filename

Except multiple patterns

grep -vE '(error|critical|warning)' filename

Show data from file without comments

grep -v ^[[:space:]]*# filename

Show data from file without comments and new lines

egrep -v '#|^$' filename

Show strings with a dash/hyphen

grep -e -- filename
grep -- -- filename
grep "\-\-" filename

Remove blank lines from a file and save output to new file

grep . filename > newfilename

Tool: perl

Search and replace (in place)

perl -i -pe's/SEARCH/REPLACE/' filename

Edit of *.conf files changing all foo to bar (and backup original)

perl -p -i.orig -e 's/\bfoo\b/bar/g' *.conf

Prints the first 20 lines from *.conf files

perl -pe 'exit if $. > 20' *.conf

Search lines 10 to 20

perl -ne 'print if 10 .. 20' filename

Delete first 10 lines (and backup original)

perl -i.orig -ne 'print unless 1 .. 10' filename

Delete all but lines between foo and bar (and backup original)

perl -i.orig -ne 'print unless /^foo$/ .. /^bar$/' filename

Reduce multiple blank lines to a single line

perl -p -i -00pe0 filename

Convert tabs to spaces (1t = 2sp)

perl -p -i -e 's/\t/  /g' filename

Read input from a file and report number of lines and characters

perl -lne '$i++; $in += length($_); END { print "$i lines, $in characters"; }' filename

Shell Tricks

When you get a shell, it is generally not very clean, but after following these steps, you will have a fairly clean and comfortable shell to work with.

  1. script /dev/null -c bash
  2. Ctrl-Z (to send it to background)
  3. stty raw -echo; fg (returns the shell to foreground)
  4. reset (to reset terminal)
  5. xterm (when asked for terminal type)
  6. export TERM=xterm; export SHELL=bash

Shell functions

Table of Contents

Domain resolve

# Dependencies:
#   - curl
#   - jq
 
function DomainResolve() {
 
  local _host="$1"
 
  local _curl_base="curl --request GET"
  local _timeout="15"
 
  _host_ip=$($_curl_base -ks -m "$_timeout" "https://dns.google.com/resolve?name=${_host}&type=A" | \
  jq '.Answer[0].data' | tr -d "\"" 2>/dev/null)
 
  if [[ -z "$_host_ip" ]] || [[ "$_host_ip" == "null" ]] ; then
 
    echo -en "Unsuccessful domain name resolution.\\n"
 
  else
 
    echo -en "$_host > $_host_ip\\n"
 
  fi
 
}

Example:

shell> DomainResolve nmap.org
nmap.org > 45.33.49.119
 
shell> DomainResolve nmap.org
Unsuccessful domain name resolution.

Get ASN

# Dependencies:
#   - curl
 
function GetASN() {
 
  local _ip="$1"
 
  local _curl_base="curl --request GET"
  local _timeout="15"
 
  _asn=$($_curl_base -ks -m "$_timeout" "http://ip-api.com/line/${_ip}?fields=as")
 
  _state=$(echo $?)
 
  if [[ -z "$_ip" ]] || [[ "$_ip" == "null" ]] || [[ "$_state" -ne 0 ]]; then
 
    echo -en "Unsuccessful ASN gathering.\\n"
 
  else
 
    echo -en "$_ip > $_asn\\n"
 
  fi
 
}

Example:

shell> GetASN 1.1.1.1
1.1.1.1 > AS13335 Cloudflare, Inc.
 
shell> GetASN 0.0.0.0
Unsuccessful ASN gathering.

Graph View

Backlinks