[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-9893":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":16,"stars7d":17,"stars30d":18,"stars90d":16,"forks30d":16,"starsTrendScore":19,"compositeScore":20,"rankGlobal":10,"rankLanguage":10,"license":21,"archived":22,"fork":22,"defaultBranch":23,"hasWiki":24,"hasPages":24,"topics":25,"createdAt":10,"pushedAt":10,"updatedAt":46,"readmeContent":47,"aiSummary":48,"trendingCount":16,"starSnapshotCount":16,"syncStatus":49,"lastSyncTime":50,"discoverSource":51},9893,"DevOps-Bash-tools","HariSekhon\u002FDevOps-Bash-tools","HariSekhon","1000+ DevOps Bash Scripts - AWS, GCP, Kubernetes, Docker, CI\u002FCD, APIs, SQL, PostgreSQL, MySQL, Hive, Impala, Kafka, Hadoop, Jenkins, GitHub, GitLab, BitBucket, Azure DevOps, TeamCity, Spotify, MP3, LDAP, Code\u002FBuild Linting, pkg mgmt for Linux, Mac, Python, Perl, Ruby, NodeJS, Golang, Advanced dotfiles: .bashrc, .vimrc, .gitconfig, .screenrc, tmux..","https:\u002F\u002Fwww.linkedin.com\u002Fin\u002FHariSekhon",null,"Shell",8295,1572,98,7,0,4,43,1,40.59,"MIT License",false,"master",true,[26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45],"api","aws","bash","ci","cloudera","devops","docker","gcp","git","github","hacktoberfest","hadoop","jenkins","kafka","kubernetes","linux","mysql","perl","postgresql","terraform","2026-06-12 02:02:14","# Hari Sekhon - DevOps Bash Tools\n\n[![GitHub stars](https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Fstars\u002Fharisekhon\u002Fdevops-bash-tools?logo=github)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fstargazers)\n[![GitHub forks](https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Fforks\u002Fharisekhon\u002Fdevops-bash-tools?logo=github)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fnetwork)\n[![LineCount](https:\u002F\u002Fsloc.xyz\u002Fgithub\u002FHariSekhon\u002FDevOps-Bash-tools\u002F?badge-bg-color=2081C2)](https:\u002F\u002Fgithub.com\u002Fboyter\u002Fscc\u002F)\n[![Cocomo](https:\u002F\u002Fsloc.xyz\u002Fgithub\u002FHariSekhon\u002FDevOps-Bash-tools\u002F?badge-bg-color=2081C2&category=cocomo)](https:\u002F\u002Fgithub.com\u002Fboyter\u002Fscc\u002F)\n[![License](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Flicense-MIT-green)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fblob\u002Fmaster\u002FLICENSE)\n[![My LinkedIn](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLinkedIn%20Profile-HariSekhon-blue?logo=data:image\u002Fsvg%2bxml;base64,PHN2ZyByb2xlPSJpbWciIGZpbGw9IiNmZmZmZmYiIHZpZXdCb3g9IjAgMCAyNCAyNCIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj48dGl0bGU+TGlua2VkSW48L3RpdGxlPjxwYXRoIGQ9Ik0yMC40NDcgMjAuNDUyaC0zLjU1NHYtNS41NjljMC0xLjMyOC0uMDI3LTMuMDM3LTEuODUyLTMuMDM3LTEuODUzIDAtMi4xMzYgMS40NDUtMi4xMzYgMi45Mzl2NS42NjdIOS4zNTFWOWgzLjQxNHYxLjU2MWguMDQ2Yy40NzctLjkgMS42MzctMS44NSAzLjM3LTEuODUgMy42MDEgMCA0LjI2NyAyLjM3IDQuMjY3IDUuNDU1djYuMjg2ek01LjMzNyA3LjQzM2MtMS4xNDQgMC0yLjA2My0uOTI2LTIuMDYzLTIuMDY1IDAtMS4xMzguOTItMi4wNjMgMi4wNjMtMi4wNjMgMS4xNCAwIDIuMDY0LjkyNSAyLjA2NCAyLjA2MyAwIDEuMTM5LS45MjUgMi4wNjUtMi4wNjQgMi4wNjV6bTEuNzgyIDEzLjAxOUgzLjU1NVY5aDMuNTY0djExLjQ1MnpNMjIuMjI1IDBIMS43NzFDLjc5MiAwIDAgLjc3NCAwIDEuNzI5djIwLjU0MkMwIDIzLjIyNy43OTIgMjQgMS43NzEgMjRoMjAuNDUxQzIzLjIgMjQgMjQgMjMuMjI3IDI0IDIyLjI3MVYxLjcyOUMyNCAuNzc0IDIzLjIgMCAyMi4yMjIgMGguMDAzeiIvPjwvc3ZnPgo=)](https:\u002F\u002Fwww.linkedin.com\u002Fin\u002FHariSekhon\u002F)\n[![GitHub Last Commit](https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Flast-commit\u002FHariSekhon\u002FDevOps-Bash-tools?logo=github)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fcommits\u002Fmaster)\n\n[![Codacy](https:\u002F\u002Fapp.codacy.com\u002Fproject\u002Fbadge\u002FGrade\u002Fdffc1bfd13404c95b5a0ab97fd47974e)](https:\u002F\u002Fwww.codacy.com\u002Fgh\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fdashboard)\n[![CodeFactor](https:\u002F\u002Fwww.codefactor.io\u002Frepository\u002Fgithub\u002Fharisekhon\u002Fdevops-bash-tools\u002Fbadge)](https:\u002F\u002Fwww.codefactor.io\u002Frepository\u002Fgithub\u002Fharisekhon\u002Fdevops-bash-tools)\n[![Quality Gate Status](https:\u002F\u002Fsonarcloud.io\u002Fapi\u002Fproject_badges\u002Fmeasure?project=HariSekhon_DevOps-Bash-tools&metric=alert_status)](https:\u002F\u002Fsonarcloud.io\u002Fdashboard?id=HariSekhon_DevOps-Bash-tools)\n[![Maintainability Rating](https:\u002F\u002Fsonarcloud.io\u002Fapi\u002Fproject_badges\u002Fmeasure?project=HariSekhon_DevOps-Bash-tools&metric=sqale_rating)](https:\u002F\u002Fsonarcloud.io\u002Fdashboard?id=HariSekhon_DevOps-Bash-tools)\n[![Reliability Rating](https:\u002F\u002Fsonarcloud.io\u002Fapi\u002Fproject_badges\u002Fmeasure?project=HariSekhon_DevOps-Bash-tools&metric=reliability_rating)](https:\u002F\u002Fsonarcloud.io\u002Fdashboard?id=HariSekhon_DevOps-Bash-tools)\n[![Security Rating](https:\u002F\u002Fsonarcloud.io\u002Fapi\u002Fproject_badges\u002Fmeasure?project=HariSekhon_DevOps-Bash-tools&metric=security_rating)](https:\u002F\u002Fsonarcloud.io\u002Fdashboard?id=HariSekhon_DevOps-Bash-tools)\n[![Vulnerabilities](https:\u002F\u002Fsonarcloud.io\u002Fapi\u002Fproject_badges\u002Fmeasure?project=HariSekhon_DevOps-Bash-tools&metric=vulnerabilities)](https:\u002F\u002Fsonarcloud.io\u002Fsummary\u002Fnew_code?id=HariSekhon_DevOps-Bash-tools)\n\n\u003C!--\nBitBucket exposes HTML comments - open issue - works properly on GitHub\u002FGitLab\ndoesn't detect shell code properly\n[![Lines of Code](https:\u002F\u002Fsonarcloud.io\u002Fapi\u002Fproject_badges\u002Fmeasure?project=HariSekhon_DevOps-Bash-tools&metric=ncloc)](https:\u002F\u002Fsonarcloud.io\u002Fdashboard?id=HariSekhon_DevOps-Bash-tools)\n-->\n\n[![Linux](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FOS-Linux-blue?logo=linux)](#hari-sekhon---devops-bash-tools)\n[![Mac](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FOS-Mac-blue?logo=apple)](#hari-sekhon---devops-bash-tools)\n[![Docker](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fcontainer-Docker-blue?logo=docker&logoColor=white)](https:\u002F\u002Fhub.docker.com\u002Fr\u002Fharisekhon\u002Fbash-tools)\n[![Dockerfile](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Frepo-Dockerfiles-blue?logo=docker&logoColor=white)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDockerfiles)\n[![DockerHub Pulls](https:\u002F\u002Fimg.shields.io\u002Fdocker\u002Fpulls\u002Fharisekhon\u002Fbash-tools?label=DockerHub%20pulls&logo=docker&logoColor=white)](https:\u002F\u002Fhub.docker.com\u002Fr\u002Fharisekhon\u002Fbash-tools)\n[![StarTrack](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FStar-Track-blue?logo=github)](https:\u002F\u002Fseladb.github.io\u002FStarTrack-js\u002F#\u002Fpreload?r=HariSekhon,Nagios-Plugins&r=HariSekhon,Dockerfiles&r=HariSekhon,DevOps-Python-tools&r=HariSekhon,DevOps-Perl-tools&r=HariSekhon,DevOps-Bash-tools&r=HariSekhon,HAProxy-configs&r=HariSekhon,SQL-scripts)\n[![StarCharts](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FStar-Charts-blue?logo=github)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fblob\u002Fmaster\u002FSTARCHARTS.md)\n\n[![Mac Homebrew](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FMac-Homebrew-999999?logo=apple&logoColor=white)](https:\u002F\u002Fbrew.sh\u002F)\n[![Alpine](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLinux-Alpine-0D597F?logo=alpine%20linux)](https:\u002F\u002Falpinelinux.org\u002F)\n[![CentOS](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLinux-CentOS-262577?logo=centos&logoColor=white)](https:\u002F\u002Fwww.centos.org\u002F)\n[![Debian](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLinux-Debian-A81D33?logo=debian)](https:\u002F\u002Fwww.debian.org\u002F)\n[![Fedora](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLinux-Fedora-294172?logo=fedora&logoColor=white)](https:\u002F\u002Fgetfedora.org\u002F)\n[![Redhat](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLinux-Redhat-EE0000?logo=red%20hat)](https:\u002F\u002Fwww.redhat.com\u002Fen)\n[![Rocky](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLinux-Rocky-10B981?logo=rockylinux&logoColor=white)](https:\u002F\u002Frockylinux.org\u002F)\n[![Ubuntu](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLinux-Ubuntu-E95420?logo=ubuntu&logoColor=white)](https:\u002F\u002Fubuntu.com\u002F)\n\n\u003C!-- TODO: fix\n[![DockerHub Build Automated](https:\u002F\u002Fimg.shields.io\u002Fdocker\u002Fautomated\u002Fharisekhon\u002Fbash-tools?logo=docker&logoColor=white)](https:\u002F\u002Fhub.docker.com\u002Fr\u002Fharisekhon\u002Fbash-tools)\n[![Docker Build Status](https:\u002F\u002Fimg.shields.io\u002Fdocker\u002Fcloud\u002Fbuild\u002Fharisekhon\u002Fbash-tools?logo=docker&logoColor=white)](https:\u002F\u002Fhub.docker.com\u002Fr\u002Fharisekhon\u002Fbash-tools\u002Fbuilds)\n-->\n\n\u003C!--\nofficial badges without logos to differentiate them\n\nthis one I don't trust it'll stick around so using shields version instead\n[![Build Status](https:\u002F\u002Fbadges.herokuapp.com\u002Ftravis\u002FHariSekhon\u002FDevOps-Bash-tools?label=Travis%20CI)](https:\u002F\u002Ftravis-ci.org\u002FHariSekhon\u002FDevOps-Bash-tools)\n\nawkward URLs more nicely replaced with shields.io\n\n[![AppVeyor](https:\u002F\u002Fci.appveyor.com\u002Fapi\u002Fprojects\u002Fstatus\u002Fu6f97cskcgb30sce\u002Fbranch\u002Fmaster?svg=true)](https:\u002F\u002Fci.appveyor.com\u002Fproject\u002FHariSekhon\u002Fdevops-bash-tools\u002Fbranch\u002Fmaster)\n[![Drone](https:\u002F\u002Fcloud.drone.io\u002Fapi\u002Fbadges\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fstatus.svg)](https:\u002F\u002Fcloud.drone.io\u002FHariSekhon\u002FDevOps-Bash-tools)\n-->\n\n[![CI Builds Overview](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FCI%20Builds-Overview%20Page-blue?logo=circleci)](https:\u002F\u002Fharisekhon.github.io\u002FCI-CD\u002F)\n[![Jenkins](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FJenkins-ready-blue?logo=jenkins&logoColor=white)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fblob\u002Fmaster\u002FJenkinsfile)\n[![Concourse](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FConcourse-ready-blue?logo=concourse&logoColor=white)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fblob\u002Fmaster\u002Fcicd\u002F.concourse.yml)\n[![GoCD](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGoCD-ready-blue?logo=go&logoColor=white)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fblob\u002Fmaster\u002Fcicd\u002F.gocd.yml)\n[![TeamCity](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTeamCity-ready-blue?logo=teamcity)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FTeamCity-CI)\n\n[![CircleCI](https:\u002F\u002Fcircleci.com\u002Fgh\u002FHariSekhon\u002FDevOps-Bash-tools.svg?style=svg)](https:\u002F\u002Fcircleci.com\u002Fgh\u002FHariSekhon\u002FDevOps-Bash-tools)\n[![BuildKite](https:\u002F\u002Fimg.shields.io\u002Fbuildkite\u002Ff11bdd9690a9bac9a8edc6094dc2f2b9af3218a7a15d4ec17d\u002Fmaster?label=BuildKite&logo=buildkite)](https:\u002F\u002Fbuildkite.com\u002Fhari-sekhon\u002Fdevops-bash-tools)\n[![AppVeyor](https:\u002F\u002Fimg.shields.io\u002Fappveyor\u002Fbuild\u002Fharisekhon\u002Fdevops-bash-tools\u002Fmaster?logo=appveyor&label=AppVeyor)](https:\u002F\u002Fci.appveyor.com\u002Fproject\u002FHariSekhon\u002Fdevops-bash-tools\u002Fbranch\u002Fmaster)\n[![Drone](https:\u002F\u002Fimg.shields.io\u002Fdrone\u002Fbuild\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fmaster?logo=drone&label=Drone)](https:\u002F\u002Fcloud.drone.io\u002FHariSekhon\u002FDevOps-Bash-tools)\n[![Codefresh](https:\u002F\u002Fg.codefresh.io\u002Fapi\u002Fbadges\u002Fpipeline\u002Fharisekhon\u002FGitHub%2FDevOps-Bash-tools?branch=master&key=eyJhbGciOiJIUzI1NiJ9.NWU1MmM5OGNiM2FiOWUzM2Y3ZDZmYjM3.O69674cW7vYom3v5JOGKXDbYgCVIJU9EWhXUMHl3zwA&type=cf-1)](https:\u002F\u002Fg.codefresh.io\u002Fpipelines\u002Fedit\u002Fnew\u002Fbuilds?id=5e53eaeea284e010982eaa6e&pipeline=DevOps-Bash-tools&projects=GitHub&projectId=5e52ca8ea284e00f882ea992&context=github&filter=page:1;pageSize:10;timeFrameStart:week)\n[![Cirrus CI](https:\u002F\u002Fimg.shields.io\u002Fcirrus\u002Fgithub\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fmaster?logo=Cirrus%20CI&label=Cirrus%20CI)](https:\u002F\u002Fcirrus-ci.com\u002Fgithub\u002FHariSekhon\u002FDevOps-Bash-tools)\n[![Semaphore](https:\u002F\u002Fharisekhon.semaphoreci.com\u002Fbadges\u002FDevOps-Bash-tools.svg)](https:\u002F\u002Fharisekhon.semaphoreci.com\u002Fprojects\u002FDevOps-Bash-tools)\n[![Buddy](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FBuddy-ready-1A86FD?logo=buddy)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fblob\u002Fmaster\u002Fbuddy.yml)\n[![Shippable](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FShippable-legacy-lightgrey?logo=jfrog&label=Shippable)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fblob\u002Fmaster\u002Fshippable.yml)\n[![Travis CI](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTravisCI-ready-blue?logo=travis&label=Travis%20CI)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fblob\u002Fmaster\u002Ftravis\u002F.travis.yml)\n[![Reviewed by Hound](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FReviewed%20by-Hound-8E64B0.svg)](https:\u002F\u002Fhoundci.com)\n\n[![Repo on GitHub](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Frepo-GitHub-2088FF?logo=github)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools)\n[![Repo on GitLab](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Frepo-GitLab-FCA121?logo=gitlab)](https:\u002F\u002Fgitlab.com\u002FHariSekhon\u002FDevOps-Bash-tools)\n[![Repo on Azure DevOps](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Frepo-Azure%20DevOps-0078D7?logo=azure%20devops)](https:\u002F\u002Fdev.azure.com\u002Fharisekhon\u002FGitHub\u002F_git\u002FDevOps-Bash-tools)\n[![Repo on BitBucket](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Frepo-BitBucket-0052CC?logo=bitbucket)](https:\u002F\u002Fbitbucket.org\u002FHariSekhon\u002FDevOps-Bash-tools)\n\n[![Azure DevOps Pipeline](https:\u002F\u002Fdev.azure.com\u002Fharisekhon\u002FGitHub\u002F_apis\u002Fbuild\u002Fstatus\u002FHariSekhon.DevOps-Bash-tools?branchName=master)](https:\u002F\u002Fdev.azure.com\u002Fharisekhon\u002FGitHub\u002F_build\u002Flatest?definitionId=1&branchName=master)\n[![GitLab Pipeline](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitLab%20CI-legacy-lightgrey?logo=gitlab)](https:\u002F\u002Fgitlab.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fpipelines)\n[![BitBucket Pipeline](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FBitbucket%20CI-legacy-lightgrey?logo=bitbucket)](https:\u002F\u002Fbitbucket.org\u002Fharisekhon\u002Fdevops-bash-tools\u002Faddon\u002Fpipelines\u002Fhome#!\u002F)\n[![AWS CodeBuild](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FAWS%20CodeBuild-ready-blue?logo=amazon%20aws)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fblob\u002Fmaster\u002Fcicd\u002Fbuildspec.yml)\n[![GCP Cloud Build](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGCP%20Cloud%20Build-ready-blue?logo=google%20cloud&logoColor=white)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fblob\u002Fmaster\u002Fcicd\u002Fcloudbuild.yaml)\n\n[![ShellCheck](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fshellcheck.yaml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fshellcheck.yaml)\n[![JSON](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fjson.yaml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fjson.yaml)\n[![YAML](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fyaml.yaml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fyaml.yaml)\n[![XML](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fxml.yaml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fxml.yaml)\n[![Markdown](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fmarkdown.yaml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fmarkdown.yaml)\n[![Validation](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fvalidate.yaml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fvalidate.yaml)\n[![Kics](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fkics.yaml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fkics.yaml)\n[![Grype](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fgrype.yaml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fgrype.yaml)\n[![Semgrep](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fsemgrep.yaml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fsemgrep.yaml)\n[![Semgrep Cloud](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fsemgrep-cloud.yaml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fsemgrep-cloud.yaml)\n[![Trivy](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Ftrivy.yaml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Ftrivy.yaml)\n\n[![Docker Build (Alpine)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fdocker_bash_alpine.yaml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fdocker_bash_alpine.yaml)\n[![Docker Build (Debian)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fdocker_bash_debian.yaml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fdocker_bash_debian.yaml)\n[![Docker Build (Fedora)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fdocker_bash_fedora.yaml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fdocker_bash_fedora.yaml)\n[![Docker Build (Ubuntu)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fdocker_bash_ubuntu.yaml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fdocker_bash_ubuntu.yaml)\n\n[![GitHub Actions Ubuntu](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fworkflows\u002FGitHub%20Actions%20Ubuntu\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions?query=workflow%3A%22GitHub+Actions+Ubuntu%22)\n[![Mac](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fmac.yaml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fmac.yaml)\n[![Mac 11](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fmac_11.yaml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fmac_11.yaml)\n[![Mac 12](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fmac_12.yaml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions\u002Fworkflows\u002Fmac_12.yaml)\n[![Ubuntu](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fworkflows\u002FUbuntu\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions?query=workflow%3A%22Ubuntu%22)\n[![Ubuntu 20.04](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fworkflows\u002FUbuntu%2020.04\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions?query=workflow%3A%22Ubuntu+20.04%22)\n[![Ubuntu 22.04](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fworkflows\u002FUbuntu%2022.04\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions?query=workflow%3A%22Ubuntu+22.04%22)\n[![Debian](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fworkflows\u002FDebian\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions?query=workflow%3A%22Debian%22)\n[![Debian 10](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fworkflows\u002FDebian%2010\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions?query=workflow%3A%22Debian+10%22)\n[![Debian 11](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fworkflows\u002FDebian%2011\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions?query=workflow%3A%22Debian+11%22)\n[![Debian 12](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fworkflows\u002FDebian%2012\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions?query=workflow%3A%22Debian+12%22)\n[![Fedora](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fworkflows\u002FFedora\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions?query=workflow%3A%22Fedora%22)\n[![Alpine](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fworkflows\u002FAlpine\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions?query=workflow%3A%22Alpine%22)\n[![Alpine 3](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fworkflows\u002FAlpine%203\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions?query=workflow%3A%22Alpine+3%22)\n\n[![Python 3.7](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fworkflows\u002FPython%203.7\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions?query=workflow%3A%22Python+3.7%22)\n[![Python 3.8](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fworkflows\u002FPython%203.8\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions?query=workflow%3A%22Python+3.8%22)\n[![Python 3.9](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fworkflows\u002FPython%203.9\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions?query=workflow%3A%22Python+3.9%22)\n[![Python 3.10](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fworkflows\u002FPython%203.10\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions?query=workflow%3A%22Python+3.10%22)\n[![Python 3.11](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fworkflows\u002FPython%203.11\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions?query=workflow%3A%22Python+3.11%22)\n\u003C!--\n[![Self Hosted](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fworkflows\u002FSelf%20Hosted\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Factions?query=workflow%3A%22Self+Hosted%22)\n-->\n\n\u003C!-- TODO: https:\u002F\u002Fcodecov.io, https:\u002F\u002Fcoveralls.io -->\n\n[git.io\u002Fbash-tools](https:\u002F\u002Fgit.io\u002Fbash-tools)\n\n1000+ DevOps Shell Scripts and Advanced Bash environment.\n\nFast, Advanced Systems Engineering, Automation, APIs, shorter CLIs, etc.\n\nHeavily used in many [GitHub repos](https:\u002F\u002Fgithub.com\u002Fsearch?o=desc&q=user%3Aharisekhon+type%3Arepository&type=Repositories), dozens of [DockerHub builds](https:\u002F\u002Fhub.docker.com\u002Fr\u002Fharisekhon) ([Dockerfiles](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDockerfiles)) and 600+ [CI builds](https:\u002F\u002Fharisekhon.github.io\u002FCI-CD\u002F).\n\n## Summary\n\n- Scripts for many popular DevOps technologies, see [Index](#index) below for more details\n- Advanced configs for common tools like [Git](https:\u002F\u002Fgit-scm.com\u002F), [vim](https:\u002F\u002Fwww.vim.org\u002F), [screen](https:\u002F\u002Fwww.gnu.org\u002Fsoftware\u002Fscreen\u002F), [tmux](https:\u002F\u002Fgithub.com\u002Ftmux\u002Ftmux\u002Fwiki), [PostgreSQL psql](https:\u002F\u002Fwww.postgresql.org\u002F) etc...\n- CI configs for most major Continuous Integration products (see [CI builds](https:\u002F\u002Fharisekhon.github.io\u002FCI-CD\u002F) page)\n- CI scripts for a drop-in framework of standard checks to run in all [CI builds](https:\u002F\u002Fharisekhon.github.io\u002FCI-CD\u002F), CI detection, accounting for installation differences across CI environments, root vs user, virtualenvs etc.\n- API scripts auto-handling authentication, tokens and other details to quickly query popular APIs with a few keystrokes just supplying the `\u002Fpath\u002Fendpoint`\n- Advanced Bash environment - `.bashrc` + `.bash.d\u002F*.sh` - aliases, functions, colouring, dynamic Git & shell behaviour enhancements, automatic pathing for installations and major languages like Python, Perl, Ruby, NodeJS, Golang across Linux distributions and Mac. See [.bash.d\u002FREADME.md](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fblob\u002Fmaster\u002F.bash.d\u002FREADME.md)\n- Installs the best systems packages -\n  [AWS CLI](https:\u002F\u002Faws.amazon.com\u002Fcli\u002F),\n  [Azure CLI](https:\u002F\u002Fdocs.microsoft.com\u002Fen-us\u002Fcli\u002Fazure\u002F?view=azure-cli-latest),\n  [GCloud SDK](https:\u002F\u002Fcloud.google.com\u002Fsdk),\n  [Digital Ocean CLI](https:\u002F\u002Fdocs.digitalocean.com\u002Freference\u002Fdoctl\u002F),\n  [Terraform](https:\u002F\u002Fwww.terraform.io\u002F),\n  [Terragrunt](https:\u002F\u002Fterragrunt.gruntwork.io\u002F),\n  [GitHub CLI](https:\u002F\u002Fgithub.com\u002Fcli\u002Fcli),\n  [Kubernetes](https:\u002F\u002Fkubernetes.io\u002F)\n  [kubectl](https:\u002F\u002Fkubernetes.io\u002Fdocs\u002Freference\u002Fkubectl\u002Foverview\u002F) &\n  [kustomize](https:\u002F\u002Fkustomize.io\u002F),\n  [Helm](https:\u002F\u002Fhelm.sh\u002F),\n  [eksctl](https:\u002F\u002Feksctl.io\u002F),\n  [Docker-Compose](https:\u002F\u002Fdocs.docker.com\u002Fcompose\u002F),\n  [jq](https:\u002F\u002Fstedolan.github.io\u002Fjq\u002F)\n  and many others... extensive package lists for servers and desktops for most major Linux distributions package managers and Mac\n  - `install\u002F` - contains many installation scripts for popular open source software and direct binary downloads from GitHub releases\n  - `configs\u002F` - contains many dot configs for common technologies like ViM, top, Screen, Tmux, MySQL, PostgreSQL etc.\n  - `setup\u002F` - contains setup scripts, package lists, extra configs, Mac OS X settings etc.\n- Utility Libraries used by many hundreds of scripts and [builds](https:\u002F\u002Fharisekhon.github.io\u002FCI-CD\u002F) across [repos](https:\u002F\u002Fgithub.com\u002Fsearch?o=desc&q=user%3Aharisekhon+type%3Arepository&type=Repositories):\n  - `.bash.d\u002F` - interactive library\n  - `lib\u002F` - scripting and CI library\n- [SQL Scripts](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FSQL-scripts) - 100+ scripts for [PostgreSQL](https:\u002F\u002Fwww.postgresql.org\u002F), [MySQL](https:\u002F\u002Fwww.mysql.com\u002F), [AWS Athena](https:\u002F\u002Faws.amazon.com\u002Fathena\u002F) + [CloudTrail](https:\u002F\u002Faws.amazon.com\u002Fcloudtrail\u002F), [Google BigQuery](https:\u002F\u002Fcloud.google.com\u002Fbigquery)\n- [Templates](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FTemplates) - templates for common programming languages and build configs\n- [Kubernetes Configs](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FKubernetes-configs) - Kubernetes YAML configs for most common scenarios, including Production Best Practices, Tips & Tricks\n\nSee Also: [similar DevOps repos](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fblob\u002Fmaster\u002FREADME.md#see-also) in other languages\n\nHari Sekhon\n\nCloud & Big Data Contractor, United Kingdom\n\n(ex-Cloudera, former Hortonworks Consultant)\n\n[![My LinkedIn](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLinkedIn%20Profile-HariSekhon-blue?logo=data:image\u002Fsvg%2bxml;base64,PHN2ZyByb2xlPSJpbWciIGZpbGw9IiNmZmZmZmYiIHZpZXdCb3g9IjAgMCAyNCAyNCIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj48dGl0bGU+TGlua2VkSW48L3RpdGxlPjxwYXRoIGQ9Ik0yMC40NDcgMjAuNDUyaC0zLjU1NHYtNS41NjljMC0xLjMyOC0uMDI3LTMuMDM3LTEuODUyLTMuMDM3LTEuODUzIDAtMi4xMzYgMS40NDUtMi4xMzYgMi45Mzl2NS42NjdIOS4zNTFWOWgzLjQxNHYxLjU2MWguMDQ2Yy40NzctLjkgMS42MzctMS44NSAzLjM3LTEuODUgMy42MDEgMCA0LjI2NyAyLjM3IDQuMjY3IDUuNDU1djYuMjg2ek01LjMzNyA3LjQzM2MtMS4xNDQgMC0yLjA2My0uOTI2LTIuMDYzLTIuMDY1IDAtMS4xMzguOTItMi4wNjMgMi4wNjMtMi4wNjMgMS4xNCAwIDIuMDY0LjkyNSAyLjA2NCAyLjA2MyAwIDEuMTM5LS45MjUgMi4wNjUtMi4wNjQgMi4wNjV6bTEuNzgyIDEzLjAxOUgzLjU1NVY5aDMuNTY0djExLjQ1MnpNMjIuMjI1IDBIMS43NzFDLjc5MiAwIDAgLjc3NCAwIDEuNzI5djIwLjU0MkMwIDIzLjIyNy43OTIgMjQgMS43NzEgMjRoMjAuNDUxQzIzLjIgMjQgMjQgMjMuMjI3IDI0IDIyLjI3MVYxLjcyOUMyNCAuNzc0IDIzLjIgMCAyMi4yMjIgMGguMDAzeiIvPjwvc3ZnPgo=)](https:\u002F\u002Fwww.linkedin.com\u002Fin\u002FHariSekhon\u002F)\n\u003Cbr>*(you're welcome to connect with me on LinkedIn)*\n\n### Quick Setup\n\nTo bootstrap, install packages and link in to your shell profile to inherit all configs, do:\n\n```bash\ncurl -L https:\u002F\u002Fgit.io\u002Fbash-bootstrap | sh\n```\n\n- Adds sourcing to `.bashrc`\u002F`.bash_profile` to automatically inherit all `.bash.d\u002F*.sh` environment enhancements for all technologies (see [Inventory](#index) below)\n- Symlinks `.*` config dotfiles to `$HOME` for [git](https:\u002F\u002Fgit-scm.com\u002F), [vim](https:\u002F\u002Fwww.vim.org\u002F), top, [htop](https:\u002F\u002Fhisham.hm\u002Fhtop\u002F), [screen](https:\u002F\u002Fwww.gnu.org\u002Fsoftware\u002Fscreen\u002F), [tmux](https:\u002F\u002Fgithub.com\u002Ftmux\u002Ftmux\u002Fwiki), [editorconfig](https:\u002F\u002Feditorconfig.org\u002F), [Ansible](https:\u002F\u002Fwww.ansible.com\u002F), [PostgreSQL](https:\u002F\u002Fwww.postgresql.org\u002F) `.psqlrc` etc. (only when they don't already exist so there is no conflict with your own configs)\n- Installs OS package dependencies for all scripts (detects the OS and installs the right RPMs, Debs, Apk or Mac HomeBrew packages)\n- Installs Python packages\n- Installs [AWS CLI](https:\u002F\u002Faws.amazon.com\u002Fcli\u002F)\n\nTo only install package dependencies to run scripts, simply `cd` to the git clone directory and run `make`:\n\n```shell\ngit clone https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools bash-tools\ncd bash-tools\nmake\n```\n\n`make install` sets your shell profile to source this repo. See [Individual Setup Parts](#individual-setup-parts) below for more install\u002Funinstall options.\n\n## Index\n\n- [Dot Configs](#dot-configs) - `.gitconfig`, `.vimrc`, `.screenrc`, `.tmux.conf`, `.toprc`, `.gitignore`...\n- [Bash Environment & Libraries](#bash-environment--libraries) - `.bashrc`, `.bash.d\u002F` interactive library, `lib\u002F` scripting library\n- [Installation Scripts](#installation-scripts) for many popular open source technologies\n- [Linux & Mac](#linux--mac) - curl OAuth \u002F JWT, LDAP, find duplicate files, SSL certificate get\u002Fvalidate, URL encoding\u002Fdecoding, Vagrant\n- [Mac & AppleScript](#mac--applescript) - Mac settings and UI automation scripts, send keystrokes, mouse clicks,\n  detect foreground app, switch app, detect locked screen or screensaver, activate screensaver, Hammerspoon system event handlers such as automatically switching audio to be able to Shazam while watching on AirPods\n- [Monitoring](#monitoring) - Grafana, Prometheus, Node Exporter, scripted collection of common Linux & Mac cli\n  monitoring stats and log locations for quick generation of vendor support tarball bundles both locally and over SSH\n- [AWS - Amazon Web Services](#aws---amazon-web-services) - AWS account summary, lots of IAM reports, CIS Benchmark config hardening, EC2, ECR, EKS, Spot termination, S3 access logging, KMS key rotation info, SSM, CloudTrail, CloudWatch billing alarm with SNS notification topic and subscription for email alerts\n- [GCP - Google Cloud Platform](#gcp---google-cloud-platform) - massive GCP auto-inventory, scripts for GCE, GKE, GCR, Secret Manager, BigQuery, Cloud SQL, Cloud Scheduler, Terraform service account creation\n- [Kubernetes](#kubernetes) - massive Kubernetes auto-inventory, cluster management scripts & tricks\n- [Docker](#docker) - Docker API, Dockerhub API, Quay.io API scripts\n- [Databases](#databases) - fast CLI wrappers, instant Docker sandboxes (PostgreSQL, MySQL, MariaDB, SQLite), [SQL scripts](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FSQL-scripts), SQL script testers against all versions of a DB, advanced `.psqlrc`\n- [Data](#data) - data tools, converters and format validators for Avro, Parquet, CSV, JSON, INI \u002F Properties files (Java), LDAP LDIF, XML, YAML\n- [Big Data & NoSQL](#big-data--nosql) - Kafka, Hadoop, HDFS, Hive, Impala, ZooKeeper, Cloudera Manager API & Cloudera Navigator API scripts\n- [Git - GitHub, GitLab, Bitbucket, Azure DevOps](#git---github-gitlab-bitbucket-azure-devops) - scripts for Git local & mirror management, GitHub, GitLab & BitBucket APIs\n- [Markdown](#markdown) - generate Markdown indexes and debug `mdl` issues like MD005 inconsistent list indentation in large `README.md` files\n- [CI\u002FCD - Continuous Integration \u002F Continuous Delivery](#cicd---continuous-integration--continuous-deployment) - API scripts & build pipeline configs for most major CI systems:\n  - Jenkins, Concourse, GoCD, TeamCity - one-touch boot & build\n  - Azure DevOps Pipelines, GitHub Actions Workflows, GitLab CI, BitBucket Pipelines, AppVeyor, BuildKite, Travis CI, Circle CI, Codefresh, CodeShip, Drone.io, Semaphore CI, Shippable ...\n  - Terraform Cloud, Octopus Deploy\n  - Checkov \u002F Bridgecrew Cloud\n- [AI & IPaaS](#ai--ipaas) - OpenAI (ChatGPT), Make.com\n- [Internet Services](#internet-services) - Google Maps, Cloudflare, DataDog, Digital Ocean, Kong API Gateway, GitGuardian, Jira, NGrok, Traefik, Pingdom, Wordpress and various pastebins and file upload sites\n- [Java](#java) - Java utilies to debug running Java programs or decompile Java JAR code for deeper debugging\n- [Python](#python) - Python utilities & library management\n- [Perl](#perl) - Perl utilities & library management\n- [Golang](#golang) - Golang utilities\n- [Diagrams](#diagrams) - scripts to generate diagrams from D2lang, MermaidJS and Python Mingrammer source code used in my [HariSekhon\u002FDiagrams-as-Code](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDiagrams-as-Code) repo\n- [Media](#media) - video downloaders & converts, MP3 metadata editing, grouping and ordering of albums and audiobooks, mkv\u002Favi to mp4 converters, 720p video downscaler for posting to social media, download YouTube videos or even entire channels and videos from other social media sites like Twitter \u002F X or Facebook, terminal gif capture\n- [Spotify](#spotify) - 40+ Spotify API scripts for backups, managing playlists, track deduplication, URI conversion, search, add\u002Fdelete, liked tracks, followed artists, top artists, top tracks etc.\n- [More Linux & Mac](#more-linux--mac) - more systems administration scripts, package installation automation\n- [Builds, Languages & Linting](#builds-languages--linting) - programming language, build system & CI linting\n- [Templates](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FTemplates) - Templates for AWS, GCP, Terraform, Docker, Jenkins, Cloud Build, Vagrant, Puppet, Python, Bash, Go, Perl, Java, Scala, Groovy, Maven, SBT, Gradle, Make, GitHub Actions, CircleCI, Jenkinsfile, Makefile, Dockerfile, docker-compose.yml etc.\n- [Kubernetes Configs](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FKubernetes-configs) - Kubernetes YAML configs for most common scenarios, including Production Best Practices, Tips & Tricks\n\n### Dot Configs\n\nTop-level dotfiles and `configs\u002F` directory:\n\n- `.*` - dot conf files for lots of common software eg. advanced `.vimrc`, `.gitconfig`, massive `.gitignore`, `.editorconfig`, `.screenrc`, `.tmux.conf` etc.\n  - `.vimrc` - contains many awesome [vim](https:\u002F\u002Fwww.vim.org\u002F) tweaks, plus hotkeys for linting lots of different file types in place, including Python, Perl, Bash \u002F Shell, Dockerfiles, JSON, YAML, XML, CSV, INI \u002F Properties files, LDAP LDIF etc without leaving the editor!\n  - `.screenrc` - fancy [screen](https:\u002F\u002Fwww.gnu.org\u002Fsoftware\u002Fscreen\u002F) configuration including advanced colour bar, large history, hotkey reloading, auto-blanking etc.\n  - `.tmux.conf` - fancy [tmux](https:\u002F\u002Fgithub.com\u002Ftmux\u002Ftmux\u002Fwiki) configuration include advanced colour bar and plugins, settings, hotkey reloading etc.\n  - [Git](https:\u002F\u002Fgit-scm.com\u002F):\n    - `.gitconfig` - advanced Git configuration\n    - `.gitignore` - extensive Git ignore of trivial files you shouldn't commit\n    - enhanced Git diffs\n    - protections against committing AWS secret keys or merge conflict unresolved files\n\n### Bash Environment & Libraries\n\nTop-level `.bashrc` and `.bash.d\u002F` directory:\n\n- `.bashrc` - shell tuning and sourcing of `.bash.d\u002F*.sh`\n- `.bash.d\u002F*.sh` - thousands of lines of advanced bashrc code, aliases, functions and environment variables for:\n  - [Linux](https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FLinux) & [Mac](https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FMacOS)\n  - SCM - [Git](https:\u002F\u002Fgit-scm.com\u002F), [Mercurial](https:\u002F\u002Fwww.mercurial-scm.org\u002F), [Svn](https:\u002F\u002Fsubversion.apache.org)\n  - [AWS](https:\u002F\u002Faws.amazon.com\u002F)\n  - [GCP](https:\u002F\u002Fcloud.google.com\u002F)\n  - [Docker](https:\u002F\u002Fwww.docker.com\u002F)\n  - [Kubernetes](https:\u002F\u002Fkubernetes.io\u002F)\n  - [Kafka](http:\u002F\u002Fkafka.apache.org\u002F)\n  - [Vagrant](https:\u002F\u002Fwww.vagrantup.com\u002F)\n  - automatic GPG and SSH agent handling for handling encrypted private keys without re-entering passwords, and lazy evaluation to only prompt key load the first time SSH is called\n  - and lots more - see [.bash.d\u002FREADME](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FDevOps-Bash-tools\u002Fblob\u002Fmaster\u002F.bash.d\u002FREADME.md) for a more detailed list\n  - run `make bash` to link `.bashrc`\u002F`.bash_profile` and the `.*` dot config files to your `$HOME` directory to auto-inherit everything\n- `lib\u002F*.sh` - Bash utility libraries full of functions for\n  [Docker](https:\u002F\u002Fwww.docker.com\u002F),\n  environment,\n  CI detection ([Travis CI](https:\u002F\u002Ftravis-ci.org\u002F), [Jenkins](https:\u002F\u002Fjenkins.io\u002F) etc),\n  port and HTTP url availability content checks etc.\n  Sourced from all my other [GitHub repos](https:\u002F\u002Fgithub.com\u002Fharisekhon) to make setting up Dockerized tests easier.\n\n### Installation Scripts\n\n- `install\u002Finstall_*.sh` - various simple to use installation scripts for common technologies like:\n  - [AWS CLI](https:\u002F\u002Faws.amazon.com\u002Fcli\u002F)\n  - [Azure CLI](https:\u002F\u002Fdocs.microsoft.com\u002Fen-us\u002Fcli\u002Fazure\u002F?view=azure-cli-latest)\n  - [GCloud SDK](https:\u002F\u002Fcloud.google.com\u002Fsdk)\n  - [GitHub CLI](https:\u002F\u002Fcli.github.com\u002F)\n  - [Terraform](https:\u002F\u002Fwww.terraform.io\u002F)\n  - [Terragrunt](https:\u002F\u002Fterragrunt.gruntwork.io\u002F)\n  - [Direnv](https:\u002F\u002Fdirenv.net\u002F)\n  - [Ansible](https:\u002F\u002Fwww.ansible.com\u002F)\n  - [K3s](https:\u002F\u002Fk3s.io)\n  - [MiniKube](https:\u002F\u002Fkubernetes.io\u002Fdocs\u002Fsetup\u002Flearning-environment\u002Fminikube\u002F) (Kubernetes)\n  - [MiniShift](https:\u002F\u002Fwww.okd.io\u002Fminishift\u002F)\n    ([Redhat OpenShift](https:\u002F\u002Fwww.openshift.com\u002F) \u002F [OKD](https:\u002F\u002Fwww.okd.io\u002F) dev VMs)\n  - [Maven](https:\u002F\u002Fmaven.apache.org\u002F)\n  - [Gradle](https:\u002F\u002Fgradle.org\u002F)\n  - [SBT](https:\u002F\u002Fwww.scala-sbt.org\u002F)\n  - [EPEL](https:\u002F\u002Ffedoraproject.org\u002Fwiki\u002FEPEL)\n  - [RPMforge](http:\u002F\u002Frepoforge.org\u002F)\n  - [Homebrew](https:\u002F\u002Fbrew.sh\u002F)\n  - [Travis CI](https:\u002F\u002Ftravis-ci.org\u002F)\n  - [Circle CI](https:\u002F\u002Fcircleci.com\u002F)\n  - [AppVeyor](https:\u002F\u002Fwww.appveyor.com\u002F)\n  - [BuildKite](https:\u002F\u002Fbuildkite.com)\n  - [Avro Tools](https:\u002F\u002Favro.apache.org\u002F)\n  - [Parquet Tools](https:\u002F\u002Fgithub.com\u002Fapache\u002Fparquet-mr\u002Ftree\u002Fmaster\u002Fparquet-tools)\n  - [Prometheus](https:\u002F\u002Fprometheus.io\u002F)\n  - various JDKs and RDBMS JDBC connector jars\n  - and many more...\n\n### Linux & Mac\n\n`bin\u002F` directory:\n\n- `login.sh` - logs to major Cloud platforms if their credentials are found in the environment, CLIs such as AWS, GCP, Azure, GitHub... Docker registries: DockerHub, GHCR, ECR, GCR, GAR, ACR, Gitlab, Quay...\n- `clean_caches.sh` - cleans out OS package and programming language caches - useful to save space or reduce Docker image size\n- `command_return_to_current_window.sh` - runs a Mac or Linux command that opens a window and then switches back to the original foreground window\n- `crypto_dice_rolls.sh` - generates 100 random dice rolls to test a new crypto hardware wallet's fidelity (do not use this for your real crypto seed as your machine could be infected with malware which steals your seed phrase)\n- `delete_duplicate_files.sh` - deletes duplicate files with (N) suffixes, commonly caused by web browser downloads,\n  in the given or current directory. Checks they're exact duplicates of a matching basename file without the (N) suffix with\n  the exact same checksum for safety. Prompts to delete per file. To auto-accept deletions, do\n  `yes | delete_duplicate_files.sh`. This is a fast way of cleaning up your `~\u002FDownloads` directory and can be put your\n  user crontab\n- `disk_speed_read_sequential_dd.sh` - runs a sequential read speed test from the given file using dd and bypassing filesystem cache for a more accurate test\n- `disk_speed_read_random_dd.sh` - runs a random I\u002FO read speed test from the given file using dd and bypassing filesystem cache for a more accurate test\n- `disk_speed_write_sequential_dd.sh` - runs a sequential write speed test to a file in the given or current directory using dd and bypassing filesystem cache for a more accurate test\n- `disk_speed_read_sequential_fio.sh` - runs a sequential read speed test in the current or given directory using fio\n- `disk_speed_read_random_fio.sh` - runs a random I\u002FO read test in the current or given directory using fio\n- `disk_speed_write_sequential_fio.sh` - runs a sequential write speed test to the current or given directory using fio\n- `disk_speed_write_random_fio.sh` - runs a sequential write speed test to the current or given directory using fio\n- `download_url_file.sh` - downloads a file from a URL using wget with no clobber and continue support, or curl with atomic replacement to avoid race conditions. Used by `github\u002Fgithub_download_release_file.sh`, `github_download_release_jar.sh`, and `install\u002Fdownload_*_jar.sh`\n- `curl_auth.sh` - shortens `curl` command by auto-loading your OAuth2 \u002F JWT API token or username & password from environment variables or interactive starred password prompt through a ram file descriptor to avoid placing them on the command line (which would expose your credentials in the process list or OS audit log files). Used by many other adjacent API querying scripts\n- `curl_with_cookies.sh` - extracts cookies for a given URL from your `\\$BROWSER`'s cookie jar and passes them to the `curl` command along with the rest of the args (workaround for older curl builds and Homebrew builds that don't have the newer `--cookies-from-browser functionality)\n- `find_duplicate_files*.sh` - finds duplicate files by size and\u002For checksum in given directory trees. Checksums are only done on files that already have matching byte counts for efficiency\n- `find_broken_links.sh` - find broken links with delays to avoid tripping defenses\n- `find_broken_symlinks.sh` - find broken symlinks pointing to non-existent files\u002Fdirectories\n- `find_lock.sh` - tries to find if a lockfile is used in the given or current working directory by taking snapshots of the file list before and after a prompt in which you should open\u002Fclose an application\n- `foreach_path_bin.sh` - runs each binary of the given name found in `$PATH` with the args given. Useful to find all the installed versions of a program in different paths eg. `~\u002Fbin\u002F` vs `\u002Fusr\u002Flocal\u002Fbin\u002F` eg. `foreach_path_bin.sh terraform --version`\n- `http_duplicate_urls.sh` - find duplicate URLs in a given web page\n- `htmldecode.sh` - decodes HTML encoding. Detects available tools such as Perl, Python or xmlstarlet and uses whatever is available\n- `ldapsearch.sh` - shortens `ldapsearch` command by inferring switches from environment variables\n- `ldap_user_recurse.sh` \u002F `ldap_group_recurse.sh` - recurse Active Directory LDAP users upwards to find all parent groups, or groups downwards to find all nested users (useful for debugging LDAP integration and group-based permissions)\n- `linux_distro_versions.sh` - quickly returns the list of major versions for a given Linux distro\n- `linux_command_return_to_current_window.sh` - runs a Linux command that opens a window and then switches back to the original foreground window\n- `mac_command_return_to_current_window.sh` - runs a Mac command that opens a window and then switches back to the original foreground window\n- `diff_line_threshold.sh` - compares two files vs a line count diff threshold to determine if they are radically different. Used to avoid overwriting files which are not mere updates but completely different files\n- `mv.sh` - moves directory trees resumably and removes the source files as they're copied over. Useful to migrate data from one disk to another, optionally with checksums. Uses rsync and shows the overall % of files transferred and the MB\u002Fs data transfer rate\n- `network_gateway.sh` - get the network gateway IP address on Linux or Mac\n- `network_randomize_mac_address.sh` - assigns a new random mac address to your network interface\n- `open.sh` - opens given arg, file or URL using whatever default system opener is available for Linux or Mac\n- `organize_downloads.sh` - moves files of well-known extensions in the `$HOME\u002FDownloads` directory older than 1 week to capitalized subdirectories of their type to keep the `$HOME\u002FDownloads\u002F` directory tidy\n- `copy_to_clipboard.sh` - copies stdin or string arg to system clipboard on Linux or Mac\n- `paste_from_clipboard.sh` - pastes from system clipboard to stdout on Linux or Mac\n- `paste_from_clipboard_upon_changes.sh` - pastes from system clipboard to stdout on Linux or Mac whenever the clipboard changes\n- `paste_diff_settings.sh` - takes snapshots of before and after clipboard changes and diffs them to show config changes\n- `processes_ram_sum.sh` - sums the RAM usage of all processes matching a given regex in GB to one decimal place\n- `pldd.sh` - parses `\u002Fproc` on Linux to show the runtime `.so` loaded dynamic shared libraries a program pid is using. Runtime equivalent of the classic static `ldd` command and because the system `pldd` command often fails to attach to a process\n- `random_select.sh` - selects one of given args at random. Useful for sampling, running randomized subsets of large test suites etc.\n- `random_number.sh` - prints a random integer between two integer arguments (inclusive)\n- `random_string.sh` - prints a random alphanumeric string of a given length\n- `screen_terminal_to_stdout.sh` - dumps the GNU Screen terminal output to stdout\n- `screen_terminal_to_clipboard.sh` - dumps the GNU Screen terminal output to a temp file and copies to clipboard for sharing & debugging purposes\n- `shields_embed_logo.sh` - base64 encodes a given icon file or url and prints the `logo=...` url parameter you need to add the [shields.io](https:\u002F\u002Fshields.io\u002F) badge url\n- `shorten_text_selection.sh` - shortens the selected text in the prior window. Replaces `and` with `&` and crushes out multiple blank lines. I use this for LinkedIn comments due to the short 1250 character limit\n- `shred_file.sh` - overwrites a file 7 times to DoD standards before deleting it to prevent recovery of sensitive information\n- `shred_free_space.sh` - overwrites free space to prevent recovery of sensitive information for files that have already been deleted\n- `split.sh` - split large files into N parts (defaults to the number of your CPU cores) to parallelize operations on them\n- `ssl_get_cert.sh` - gets a remote `host:port` server's SSL cert in a format you can pipe, save and use locally, for example in Java truststores\n- `ssl_verify_cert.sh` - verifies a remote SSL certificate (battle tested more feature-rich version `check_ssl_cert.pl` exists in the [Advanced Nagios Plugins](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FNagios-Plugins) repo)\n- `ssl_verify_cert_by_ip.sh` - verifies SSL certificates on specific IP addresses, useful to test SSL source addresses for CDNs, such as Cloudflare Proxied sources before enabling SSL Full-Strict Mode for end-to-end, or Kubernetes ingresses (see also `curl_k8s_ingress.sh`)\n- `text_filter_ending_substrings.sh` - for a given patterns file of substring endings, print all lines that match in the following files. Uses awk to safely handle all characters as literals, unlike grep, while also maintaining end anchoring which you cannot do using `grep -F`. Optimized awk code uses a bucketing hash for performance to not attempt matching lines which are shorter than patterns, reducing the number of match attempts\n- `tmux_vertical.sh` - launches tmux with N-way vertical shell split or commands given as args in equally balanced vertical panes. Fast way to launch a bunch of shell or commands in an easily reviewable side-by-side way\n- `tmux_horizontal.sh` - same as above but split horizontally\n- `tmux_square.sh` - same as above but with 4 panes in a square tiled view\n- `urlencode.sh` \u002F `urldecode.sh` - URL encode\u002Fdecode quickly on the command line, in pipes etc.\n- `urlextract.sh` - extracts the URLs from a given string arg, file or standard input\n- `url_extract_redirects.sh` - extracts the URLs from a given string arg, file or standard input, queries each one and outputs the redirected urls instead to stdout\n- `url_replace_redirects.sh` - extracts the URLs from a given string arg, file or standard input, queries each one and outputs the entire contents to stdout with the urls replaced by the redirected urls\n- `urlopen.sh` - opens the URL given as an arg, or first URL found from stdin or a given file.\n  Uses the system's default browser\n- `vagrant_hosts.sh` - generate `\u002Fetc\u002Fhosts` output from a `Vagrantfile`\n- `vagrant_total_mb.sh` - calculate the RAM committed to VMs in a `Vagrantfile`\n\nSee also [Knowledge Base notes for Linux](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FKnowledge-Base\u002Fblob\u002Fmain\u002Flinux.md)\nand [Mac](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FKnowledge-Base\u002Fblob\u002Fmain\u002Fmac.md).\n\n### Mac & AppleScript\n\nMac automation scripts to automate the Mac UI and settings\n\n`bin\u002F` directory:\n\n- `mac_diff_settings.sh` - takes before and after snapshots of UI setting changes and diffs them to make it easy to find `defaults` keys to add to `setup\u002Fmac_settings.sh` to save settings\n- `mac_restore_file.sh` - checks all the backup mount points for the latest backup that has a given file and then restores it\n- `mac_backup_du_in_progress.sh` - find large files in the currently in-progress Time Machine backup to find out what is taking so long and racking up so many more GB of changes than you expect. This helps discover large but unnecessary files that you might want to exclude using the adjacent script `mac_backup_exclude_paths.sh`\n- `mac_backup_exclude_paths.sh` - excludes many common large caches, docker and VM paths from macOS Time Machine backups\n- `mac_backup_find_excluded_paths.sh` - does a deep search for macOS Time Machine excluded backup paths on file\u002Ffolder attributes. See [HariSekhon\u002FKnowledge-Base Mac page](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FKnowledge-Base\u002Fblob\u002Fmain\u002Fmac.md#time-machine) for why\n- `mac_command_return_to_current_window.sh` - runs a Mac command that opens a window and then switches back to the original foreground window\n- `mac_rmdir.sh` - safely delete a directory on Mac only if it is empty of actual data, by first removing macOS hidden metadata files and dirs such as `.fseventsd\u002F`, `.Spotlight-V100\u002F` and `.DS_Store` - straight `rmdir` fails otherwise\n- `mac_iso_to_usb.sh` - converts a given ISO file to a USB bootable image and burns it onto a given or detected inserted USB drive\n- `mac_ramdisk.sh` - creates a mac ramdisk of given MB size. Useful for performance, or even testing disk write scripts such as `disk_speed_write_*.sh` without wearing out your SSD\n- `mac_delete_local_snapshots.sh` - deletes local macOS snapshots to free up disk space. When there is a substantial discrepancy between what the `df -h` command and the Finder UI shows, this is often the cause\n- `copy_to_clipboard.sh` - copies stdin or string arg to system clipboard on Linux or Mac\n- `paste_from_clipboard.sh` - pastes from system clipboard to stdout on Linux or Mac\n- `paste_from_clipboard_upon_changes.sh` - pastes from system clipboard to stdout on Linux or Mac whenever the clipboard changes\n- `paste_diff_settings.sh` - Takes snapshots of before and after clipboard changes and diffs them to show config changes\n\n`applescript\u002F` directory:\n\n- `keystrokes.sh` - send N keystroke combinations\n- `mouse_clicks.sh` - send N mouse click combinations to sequence of screen coordinates\n  - `get_mouse_coordinates.sh` - print the current mouse coordinates - to know what to pass to above script\n- `mouse_clicks_remote_desktop.sh` - switches to Microsoft Remote Desktop, waits 10 seconds and then clicks the mouse\n  once a minute to prevent the screensaver from coming on. Workaround to Active Directory Group Policies that don't let\n  you disable the screensaver. Point your mouse to an area that will have no mouse click effect, the Cmd-Tab to Terminal\n  and run this\n- `get_frontmost_process_title.scpt` - detect the frontmost window\n  - to detect if you should send keystrokes \u002F mouse clicks)\n- `set_frontmost_process.scpt` - switch to bring the given app to the foreground to send keystrokes \u002F mouse clicks to it\n  - `browser_get_default.scpt` - get the default configured browser in format passable to Applescript (for  above script)\n- `is_screen_locked.py` - detect if the screen is locked to stop sending keystrokes or mouse clicks\n- `is_screensaver_running.scpt` - detect if the screensaver is running to stop sending keystrokes or mouse clicks\n- `reopen_app.sh` - relaunch a given app\n  (used to reload Shazam to detect DB changes after removing tracks programmatically from its DB)\n- `spotify_app_search.sh` - runs a search in the Spotify App on Mac using Applescript\n- `spotify_app_search_filelist.sh` - searches the Spotify App for each track in a file, prompting to continue to the next track\n- `shazam_app_dump_tracks.sh` - dumps `artist - track` one per line from the Shazam local sqlite DB\n- `shazam_app_delete_track.sh` - deletes a given `\"artist\" \"track\"` from the Shazam local sqlite DB\n- `shazam_search_spotify_then_delete_track.sh` - searches for each Shazam'd track in the local Spotify desktop app,\n  then prompts to delete each track from the local Shazam DB once you've saved it in Spotify.\n  Useful to migrate Shazam'd tracks to Spotify after Apple removed the integration\n- `screensaver_activate.scpt` - activate screensaver\n- `shorten_text_selection.scpt` - shortens the selected text in the prior window. Replaces `and` with `&` and crushes\n  out multiple blank lines. I use this for LinkedIn comments due to the short 1250 character limit\n- `start_app_at_login.sh` - adds an app to the Login items to auto-start\n\nHammerspoon code has been moved to its own repo:\n\n[![Readme Card](https:\u002F\u002Fgithub-readme-stats.vercel.app\u002Fapi\u002Fpin\u002F?username=HariSekhon&repo=Hammerspoon&theme=ambient_gradient&description_lines_count=3)](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FHammerspoon)\n\nSee also [Mac](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FKnowledge-Base\u002Fblob\u002Fmain\u002Fmac.md) page\nin [HariSekhon\u002FKnowledge-Base](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FKnowledge-Base).\n\n### Monitoring\n\n`monitoring\u002F` directory:\n\n- `dump_stats.sh` - dumps common command outputs to text files in a local tarball. Useful to collect support information\n  for vendor support cases\n- `grafana_api.sh` - queries the [Grafana](https:\u002F\u002Fgrafana.com\u002F) API with authentication\n- `log_timestamp_large_intervals.sh` - finds log lines whose timestamp intervals exceed the given number of seconds and\n  outputs those log lines with the difference between the last and current timestamps. Useful to find actions that are\n  taking a long time from log files such as CI\u002FCD logs\n- `prometheus.sh` - starts [Prometheus](https:\u002F\u002Fprometheus.io\u002F) locally, downloading it if not found in `$PATH`\n- `prometheus_docker.sh` - starts [Prometheus](https:\u002F\u002Fprometheus.io\u002F) in Docker using `docker-compose`\n- `prometheus_node_exporter.sh` - starts Prometheus `node_exporter` locally, downloading it if not found in `$PATH`\n- `ssh_dump_stats.sh` - uses SSH and `dump_stats.sh` to dump common command outputs from remote servers to a local\n  tarball. Useful for vendor support cases\n- `ssh_dump_logs.sh` - Uses SSH to dump logs from server to local text files for uploading to vendor support cases\n\nSee doc pages in [HariSekhon\u002FKnowledge-Base](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FKnowledge-Base) on Grafana,\nPrometheus, OpenTSDB, InfluxDB etc.\n\n### Databases\n\n`mysql\u002F`, `postgres\u002F`, `sql\u002F` and `bin\u002F` directories:\n\n- [sql\u002F](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FSQL-scripts) - 100+ SQL scripts for [PostgreSQL](https:\u002F\u002Fwww.postgresql.org\u002F), [MySQL](https:\u002F\u002Fwww.mysql.com\u002F), [Google BigQuery](https:\u002F\u002Fcloud.google.com\u002Fbigquery) and [AWS Athena](https:\u002F\u002Faws.amazon.com\u002Fathena\u002F) [CloudTrail](https:\u002F\u002Faws.amazon.com\u002Fcloudtrail\u002F) logs integration\n- `sqlite.sh` - one-touch [SQLite](https:\u002F\u002Fwww.sqlite.org\u002Findex.html), starts sqlite3 shell with sample 'chinook' database loaded\n- `mysql*.sh` - [MySQL](https:\u002F\u002Fwww.mysql.com\u002F) scripts:\n  - `mysql.sh` - shortens `mysql` command to connect to [MySQL](https:\u002F\u002Fwww.mysql.com\u002F) by auto-populating switches from both standard environment variables like `$MYSQL_TCP_PORT`, `$DBI_USER`, `$MYSQL_PWD` (see [doc](https:\u002F\u002Fdev.mysql.com\u002Fdoc\u002Frefman\u002F8.0\u002Fen\u002Fenvironment-variables.html)) and other common environment variables like `$MYSQL_HOST` \u002F `$HOST`, `$MYSQL_USER` \u002F `$USER`, `$MYSQL_PASSWORD` \u002F `$PASSWORD`, `$MYSQL_DATABASE` \u002F `$DATABASE`\n  - `mysql_foreach_table.sh` - executes a SQL query against every table, replacing `{db}` and `{table}` in each iteration eg. `select count(*) from {table}`\n  - `mysql_*.sh` - various scripts using `mysql.sh` for row counts, iterating each table, or outputting clean lists of databases and tables for quick scripting\n  - `mysqld.sh` - one-touch [MySQL](https:\u002F\u002Fwww.mysql.com\u002F), boots docker container + drops in to `mysql` shell, with `\u002Fsql` scripts mounted in container for easy sourcing eg. `source \u002Fsql\u002F\u003Cname>.sql`. Optionally loads sample 'chinook' database\n  - see also the [SQL Scripts](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FSQL-scripts) repo for many more straight MySQL SQL scripts\n- `mariadb.sh` - one-touch [MariaDB](https:\u002F\u002Fmariadb.org\u002F), boots docker container + drops in to `mysql` shell, with `\u002Fsql` scripts mounted in container for easy sourcing eg. `source \u002Fsql\u002F\u003Cname>.sql`. Optionally loads sample 'chinook' database\n- `postgres*.sh` \u002F `psql.sh` - [PostgreSQL](https:\u002F\u002Fwww.postgresql.org\u002F) scripts:\n  - `postgres.sh` - one-touch [PostgreSQL](https:\u002F\u002Fwww.postgresql.org\u002F), boots docker container + drops in to `psql` shell, with `\u002Fsql` scripts mounted in container for easy sourcing eg. `\\i \u002Fsql\u002F\u003Cname>.sql`. Optionally loads sample 'chinook' database\n  - `psql.sh` - shortens `psql` command to connect to [PostreSQL](https:\u002F\u002Fwww.postgresql.org\u002F) by auto-populating switches from environment variables, using both standard postgres supported environment variables like `$PG*` (see [doc](https:\u002F\u002Fwww.postgresql.org\u002Fdocs\u002F12\u002Flibpq-envars.html)) as well as other common environment variables like `$POSTGRESQL_HOST` \u002F `$POSTGRES_HOST` \u002F `$HOST`, `$POSTGRESQL_USER` \u002F `$POSTGRES_USER` \u002F `$USER`, `$POSTGRESQL_PASSWORD` \u002F `$POSTGRES_PASSWORD` \u002F `$PASSWORD`, `$POSTGRESQL_DATABASE` \u002F `$POSTGRES_DATABASE` \u002F `$DATABASE`\n  - `postgres_foreach_table.sh` - executes a SQL query against every table, replacing `{db}`, `{schema}` and `{table}` in each iteration eg. `select count(*) from {table}`\n  - `postgres_*.sh` - various scripts using `psql.sh` for row counts, iterating each table, or outputting clean lists of databases, schemas and tables for quick scripting\n  - `checks\u002Fcheck_sqlfluff.sh` - recursively iterates all SQL code files found in the given or current directory and runs SQLFluff linter against them, inferring the different SQL dialects from each path\u002Ffilename\u002Fextension\n\n### AWS - Amazon Web Services\n\n`aws\u002F` directory:\n\n- [AWS](https:\u002F\u002Faws.amazon.com\u002F) scripts - `aws_*.sh`:\n  - `aws_profile.sh` - switches to an AWS Profile selected from a convenient interactive menu list of AWS profiles from `$AWS_CONFIG_FILE` - useful when you have lots of AWS work profiles\n    - see also [HariSekhon\u002FEnvironments](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FEnvironments) for automated switching using direnv when `cd`ing into relevant directories\n  - `aws_cli_create_credential.sh` - creates an AWS service account user for CI\u002FCD or CLI with Admin permissions (or other group or policy), creates an AWS Access Key, saves a credentials CSV and even prints the shell export commands and aws credentials file config to configure your environment to start using it. Useful trick to avoid CLI reauth to `aws sso login` every day.\n  - `aws_terraform_create_credential.sh` - creates a AWS terraform service account with Administrator permissions for Terraform Cloud or other CI\u002FCD systems to run Terraform plan and apply, since no CI\u002FCD systems can work with AWS SSO workflows. Stores the access key as both CSV and prints shell export commands and credentials file config as above\n  - `.envrc-aws` - copy to `.envrc` for [direnv](https:\u002F\u002Fdirenv.net\u002F) to auto-load AWS configuration settings such as AWS Profile, Compute Region, EKS cluster kubectl context etc.\n    - calls `.envrc-kubernetes` to set the `kubectl` context isolated to current shell to prevent race conditions between shells and scripts caused by otherwise naively changing the global `~\u002F.kube\u002Fconfig` context\n  - `aws_sso_ssh.sh` - launches local AWS SSO authentication pop-up (if not already authenticated), then scp's the latest resultant `~\u002F.aws\u002Fsso\u002Fcache\u002F` file to the remote server and SSH's there so that you can use AWS CLI or kubectl to EKS remotely on that server easily, without having to copy and paste the token from remote aws sso login to your local web browser\n  - `aws_terraform_create_s3_bucket.sh` - creates a Terraform S3 bucket for storing the backend state, locks out public access, enables versioning, encryption, and locks out Power Users role and optionally any given user\u002Fgroup\u002Frole ARNs via a bucket policy for safety\n  - `aws_terraform_create_dynamodb_table.sh` - creates a Terraform locking table in DynamoDB for use with the S3 backend, plus custom IAM policy which can be applied to less privileged accounts\n  - `aws_terraform_create_all.sh` - runs all of the above, plus also applies the custom DynamoDB IAM policy to the user to ensure if the account is less privileged it can still get the Terraform lock (useful for GitHub Actions environment secret for a read only user to generate Terraform Plans in Pull Request without needing approval)\n  - `aws_terraform_iam_grant_s3_dynamodb.sh` - creates IAM policies to access any S3 buckets and DynamoDB tables with `terraform-state` or `tf-state` in their names, and attaches them to the given user. Useful for limited permissions CI\u002FCD accounts that run Terraform Plan eg. in GitHub Actions pull requests\n  - `aws_account_summary.sh` - prints AWS account summary in `key = value` pairs for easy viewing \u002F grepping of things like `AccountMFAEnabled`, `AccountAccessKeysPresent`, useful for checking whether the root account has MFA enabled and no access keys, comparing number of users vs number of MFA devices etc. (see also `check_aws_root_account.py` in [Advanced Nagios Plugins](https:\u002F\u002Fgithub.com\u002FHariSekhon\u002FNagios-Plugins))\n  - `aws_billing_alarm.sh` - creates a [CloudWatch](https:\u002F\u002Faws.amazon.com\u002Fcloudwatch\u002F) billing alarm and [SNS](https:\u002F\u002Faws.amazon.com\u002Fsns\u002F) topic with subscription to email you when you incur charges above a given threshold. This is often the first thing you want to do on an account\n  - `aws_budget_alarm.sh` - creates an [AWS Budgets](https:\u002F\u002Faws.amazon.com\u002Fcloudwatch\u002F) billing alarm and [SNS](https:\u002F\u002Faws.amazon.com\u002Fsns\u002F) topic with subscription to email you when both when you start incurring forecasted charges of over 80% of your budget, and 90% actual usage. This is often the first thing you want to do on an account\n  - `aws_batch_stale_jobs.sh` - lists [AWS Batch](https:\u002F\u002Faws.amazon.com\u002Fbatch\u002F) jobs that are older than N hours in a given queue\n  - `aws_batch_kill_stale_jobs.sh` - finds and kills [AWS Batch](https:\u002F\u002Faws.amazon.com\u002Fbatch\u002F) jobs that are older than N hours in a given queue\n  - `aws_cloudfront_distribution_for_origin.sh` - returns the AWS CloudFront ARN of the distribution which serves origins containing a given substring. Useful for quickly finding the CloudFront ARN needed to give permissions to a private S3 bucket exposed via CloudFront\n  - `aws_cloudtrails_cloudwatch.sh` - lists [Cloud Trails](https:\u002F\u002Faws.amazon.com\u002Fcloudtrail\u002F) and their last delivery to [CloudWatch](https:\u002F\u002Faws.amazon.com\u002Fcloudwatch\u002Ffeatures\u002F) Logs (should be recent)\n  - `aws_cloudtrails_event_selectors.sh` - lists [Cloud Trails](https:\u002F\u002Faws.amazon.com\u002Fcloudtrail\u002F) and their event selectors to check each one has at least one event selector\n  - `aws_cloudtrails_s3_accesslogging.sh` - lists [Cloud Trails](https:\u002F\u002Faws.amazon.com\u002Fcloudtrail\u002F) buckets and their Access Logging prefix and target bucket. Checks [S3 access logging](https:\u002F\u002Fdocs.aws.amazon.com\u002FAmazonS3\u002Flatest\u002Fdev\u002FServerLogs.html) is enabled\n  - `aws_cloudtrails_s3_kms.sh` - lists [Cloud Trails](https:\u002F\u002Faws.amazon.com\u002Fcloudtrail\u002F) and whether their [S3](https:\u002F\u002Faws.amazon.com\u002Fs3\u002F) buckets are [KMS](https:\u002F\u002Faws.amazon.com\u002Fkms\u002F) secured\n  - `aws_cloudtrails_status.sh` - lists [Cloud Trails](https:\u002F\u002Faws.amazon.com\u002Fcloudtrail\u002F) status - if logging, multi-region and log file validation enabled\n  - `aws_config_all_types.sh` - lists [AWS Config](https:\u002F\u002Faws.amazon.com\u002Fconfig\u002F) recorders, checking all resource types are supported (should be true) and includes global resources (should be true)\n  - `aws_config_recording.sh` - lists [AWS Config](https:\u002F\u002Faws.amazon.com\u002Fconfig\u002F) recorders, their recording status (should be true) and their last status (should be success)\n  - `aws_csv_creds.sh` - prints AWS credentials from a CSV file as shell export statements. Useful to quickly switch your shell to some exported credentials from a service account for testing permissions or pipe to upload to a CI\u002FCD system via an API (eg. `jenkins_cred_add*.sh`, `github_actions_repo*_set_secret.sh`, `gitlab_*_set_env_vars.sh`, `circleci_*_set_env_vars.sh`, `bitbucket_*_set_env_vars.sh`, `terraform_cloud_*_set_vars.sh`, `kubectl_kv_to_secret.sh`). Supports new user and new access key csv file formats.\n  - `aws_codecommit_csv_creds.sh` - prints AWS [CodeCommit](https:\u002F\u002Faws.amazon.com\u002Fcodecommit\u002F) Git credentials from a CSV file as shell export statements. Similar use case and chaining as above\n  - `aws_ec2_*.sh` - AWS [EC2](https:\u002F\u002Faws.amazon.com\u002Fec2\u002F) scripts:\n    - `aws_ec2_instance_*.sh` - AWS EC2 [Instance](https:\u002F\u002Fdocs.aws.amazon.com\u002FAWSEC2\u002Flatest\u002FUserGuide\u002FInstances.html) scripts:\n      - `aws_ec2_instance_name_to_id.sh` - looks up an EC2 instance ID from an instance name with extra safety checks that only a single instance ID is returned and a reverse lookup on that instance ID to re-verify it matches the name. This level of safety is important when wanting to terminate an EC2 instance by name. If an instance ID is passed, returns it as is for convenience. Used by adjacent scripts\n      - `aws_ec2_instances.sh` - lists AWS EC2 instances, their DNS names and States in an easy to read table output\n      - `aws_ec2_instance_ip.sh` - determines an EC2 instance IP address, trying first for a public IP, or failing that a private IP\n      - `aws_ec2_instance_clone.sh` - clones an AWS EC2 instance by creating an AMI from the original and then booting a new instance from the AMI with the same settings as the original instance. Useful to testing risky things on a separate EC2 instance, such as Server Administrator recovery of Tableau\n      - `aws_ec2_instance_wait_for_ready.sh` - polls an AWS EC2 instance and waits for it to finish initializing to a ready state. Used by adjacent scripts\n      - `aws_ec2_instance_terminate_by_name.sh` - terminate an AWS EC2 instance by name for convenience, resolves its instance ID, verifies unique and then terminates by ID\n    - `aws_ec2_ami*.sh` - AWS EC2 [AMI](https:\u002F\u002Fdocs.aws.amazon.com\u002FAWSEC2\u002Flatest\u002FUserGuide\u002FAMIs.html) scripts:\n      - `aws_ec2_amis.sh` - list AWS EC2 AMIs belonging to your account in an easy to read table output\n      - `aws_ec2_ami_ids.sh` - lists AWS EC2 AMI IDs only, one per line, to be used in adjacent scripts that creating mapping tables and translate AMI IDs to names in inventory scripts `aws_info_ec2*.sh`\n      - `aws_ec2_ami_name_to_id.sh` - looks up an EC2 AMI ID from a name with extra safety checks that only a single AMI ID is returned and a reverse lookup on that AMI ID to re-verify it matches the name\n      - `aws_ec2_ami_boot.sh` - boots a personal EC2 instance of a given AMI for testing\n      - `aws_ec2_ami_boot_ssh.sh` - boots a personal EC2 instance of a given AMI, determines the public or private IP, and drops you into an SSH shell\n      - `aws_ec2_ami_create_from_instance.sh` - creates an AWS EC2 AMI from an EC2 instance and waits for it to become available for use\n      - `aws_ec2_ami_share_to_account.sh` - shares an AMI with another AWS account. Can specify AMI by name or id\n    - `aws_ec2_ebs_*.sh` - AWS EC2 [EBS](https:\u002F\u002Faws.amazon.com\u002Febs\u002F) scripts:\n      - `aws_ec2_ebs_volumes.sh` - list EC2 instances and their EBS volumes in the current region\n      - `aws_ec2_ebs_create_snapshot_and_wait.sh - creates a snapshot of a given EBS volume ID and waits for it to complete with exponential backoff\n      - `aws_ec2_ebs_resize_and_wait.sh - resizes an EBS volume and waits for it to complete modifying and optionally optimizing with exponential backoff\n      - `aws_ec2_ebs_volumes_unattached.sh` - list an unattached EBS volumes in a table format\n    - `aws_ec2_launch_templates_ami_id.sh` - for each Launch Template lists the AMI ID of the latest version. Useful to check EKS upgrades of node groups via Terragrunt have taken effect\n  - `aws_ecr_*.sh` - AWS [ECR](https:\u002F\u002Faws.amazon.com\u002Fecr\u002F) docker image management scripts:\n    - `aws_ecr_docker_login.sh` - authenticates Docker to AWS ECR, inferring the ECR registry from the current AWS Account ID and Region\n    - `aws_ecr_docker_build_push.sh` - builds a docker image and pushes it to ECR with not just the `latest` docker tag but also the current Git hashref and Git tags\n    - `aws_ecr_list_repos.sh` - lists ECR repos, and their docker image mutability and whether image scanning is enabled\n    - `aws_ecr_list_tags.sh` - lists all the tags for a given ECR docker image\n    - `aws_ecr_newest_image_tags.sh` - lists the tags for the given ECR docker image with the newest creation date (can use this to determine which image version to tag as `latest`)\n    - `aws_ecr_alternate_tags.sh` - lists all the tags for a given ECR docker `image:tag` (use arg `\u003Cimage>:latest` to see what version \u002F build hashref \u002F date tag has been tagged as `latest`)\n    - `aws_ecr_tag_image.sh` - tags an ECR image with another tag without pulling and pushing it\n    - `aws_ecr_tag_image_by_digest.sh` - same as above but tags an ECR image found via digest (more accurate as reference by existing tag can be a moving target). Useful to recover images that have become untagged\n    - `aws_ecr_tag_latest.sh` - tags a given ECR docker `image:tag` as `latest` without pulling or pushing the docker image\n    - `aws_ecr_tag_branch.sh` - tags a given ECR `image:tag` with the current Git branch without pulling or pushing the docker image\n    - `aws_ecr_tag_datetime.sh` - tags a given ECR docker image with its creation date and UTC timestamp (when it was uploaded to ECR) without pulling or pushing the docker image\n    - `aws_ecr_tag_newest_image_as_latest.sh` - finds and tags the newest build of a given ECR docker image as `latest` without pulling or pushing the docker image\n    - `aws_ecr_tags_timestamps.sh` - lists all the tags and their timestamps for a given ECR docker image\n    - `aws_ecr_tags_old.sh` - lists tags old","HariSekhon\u002FDevOps-Bash-tools 是一个包含超过1000个 DevOps Bash 脚本的集合，涵盖了从云服务（如AWS、GCP）到容器技术（如Kubernetes、Docker）、CI\u002FCD流程、数据库管理（如PostgreSQL、MySQL）、大数据处理（如Hadoop、Kafka）等多个领域。该项目支持多种编程语言环境下的包管理和代码构建检查，并提供了高级配置文件模板，如.bashrc、.vimrc等，极大提升了开发效率和系统管理的便捷性。适用于需要快速部署或自动化运维任务的各种规模企业及个人开发者，尤其适合在多云环境和复杂IT架构中使用。",2,"2026-06-11 03:25:16","top_topic"]