HACKER Q&A
📣 fastily

Can I see your cheatsheet?


Almost everyone I know keeps a list of (easily forgettable) command line snippets somewhere. I can't imagine that HN folks would be any different :)

So that said, could I please see your cheatsheet?

I'll go first: https://github.com/fastily/cheatsheet


  👤 asicsp Accepted Answer ✓
I use my ebooks for reference:

* GNU grep and ripgrep (https://learnbyexample.github.io/learn_gnugrep_ripgrep/)

* GNU sed (https://learnbyexample.github.io/learn_gnused/)

* GNU awk (https://learnbyexample.github.io/learn_gnuawk/)

* Ruby one-liners cookbook (https://learnbyexample.github.io/learn_ruby_oneliners/)

* Perl one-liners cookbook (https://learnbyexample.github.io/learn_perl_oneliners/)

* Command line text processing with GNU Coreutils (https://learnbyexample.github.io/cli_text_processing_coreuti...)

* Command line text processing with Rust tools (https://learnbyexample.github.io/cli_text_processing_rust/) — work-in-progress

* Computing from the Command Line (https://learnbyexample.github.io/cli-computing/) — work-in-progress


👤 CraigJPerry
Set a huuuuuuuge shell history https://github.com/craigjperry2/dotfiles/blob/aa77ddcbde63bf... then fzf ctrl+r bindings mean you can recall anything right where you need it.

If you’re going to do this then have an escape hatch for commands you don’t want memorised https://github.com/craigjperry2/dotfiles/blob/aa77ddcbde63bf...


👤 ww520

  - Find all files modified in the last 90 minutes  
    find . -type f -mmin -90  
  - Find all files and do something to each one
    find . -type f -mmin -90 | xargs ls -l  
  - Find all files modified before 2 days ago  
    find . -type f -mtime +2  
  - Removing files older than 7 days  
    find . -type f -mtime +7 -name '*.gz' -exec rm {} \;  
  - Find all files modified after last 2 days  
    find . -type f -mtime -2  
  - Find all files modified at the last 2 day mark  
    find . -type f -mtime 2

👤 mosselman
http://cht.sh/ - I use this a lot. It outputs a short summary of common commands for many cli tools. You can curl to it https://cht.sh/grep for example or install their cli client

👤 scrapheap
No maintained cheat sheet here either. I do use `history | grep` quite a lot though.

One thing I try to do is to keep my keyboard shortcuts consistent between programs/systems. That way the chance of my instinct of pressing `+a` to move the cursor to the start of the line will be correct more often.

Another thing I try to do when a project has a set of complex commands that are needed, is to add them as a rule in its Makefile, that way they're available for my colleagues as well.


👤 hillacious
One of the most useful cheat sheet utilities I've come across is cheat.sh: https://github.com/chubin/cheat.sh

If you don't have it installed it can easily be queried with curl from the command line like: `curl cheat.sh/sed`. The payload is colorized and gives a lot of examples of usage of the command. You can also query "cht.sh" to use fewer characters. There is actually a lot more advanced usage of the tool/service if you check out the README.

Apologies if this isn't what you're looking for.


👤 tiborsaas
This surprised me that people do this. I don't have any, since if I have to look up one, it's not really useful :)

But my cheat sheet is google, I usually find what I want in no time.


👤 McDyver
Delete empty files / empty directories

find . -type f -empty -print -delete

find . -type d -empty -print -delete

Move photos/videos to folder, based on their creation date

exiftool -d /home/user/Photos/%Y-%m-%d "-directory(or leave the extension out, and all files with EXIF data will be included)


👤 pcthrowaway
My only real cheatsheet is a list of shortcuts I maintain for programs/apps I use:

    system:
    ctrl+space=switch keyboard input language
    
    iterm2:
    cmd+; = autocomplete
    fn+cmd+< = scroll to top of buffer
    fn+cmd+> = scroll to bottom of buffer
    fn+cmd+^ = pgup buffer
    fn+cmd+v = pgdown buffer
    fn+< = beginning of line
    fn+> = end of line
    shift+cmd+[ = previous tab
    shift+cmd+] = next tab
    cmd+K = delete til end of line
    cmd+enter = full-screen on/off
    cmd+shift+e = show command timeline!!!
    
    chrome:
    opt+cmd+J = devtools
    cmd+] = devtools tab right
    cmd+[ = devtools tab left
    opt+space = start/stop toggl
    fn+cmd+< = move tab left
    fn+cmd+> = move tab right
    fn+cmd+^ = move tab to start
    fn+cmd+v = move tab to end
    
    sublime:
    ctrl+` = open python console
    cmd+shift+P = open command palette
    cmd+P = search open files
    ctrl+opt+C = open in browser with View in Browser package (create tmpfile if dirty)
    cmd+shift+[ = Move to left pane
    cmd+shift+] = Move to right pane
    ctrl+0 = move to sidebar
    cmd+k+b = show/hide sidebar
    cmd+k+u = uppercase
    cmd+k+l = lowercase
    ctrl+shift+v = multiple cursor down
    ctrl+shift+^ = multiple cursor up
    shift+tab = back up tab indentation (at beginning of line)
    cmd+[ = dedent (when highlighted)
    cmd+[ = indent (when highlighted)
    cmd+shift+. = show dotfiles in open file view
    cmd+shit+g (when opening file) = navigate to path for file selection
    
    # edit package override files in /User/user/Library/Application Support/Sublime Text 3/Packages/User/
    
    VS Code:
    option+^ = drag line up
    option+v = drag line down
    ctrl+` = open terminal
    cmd+p, :# = go to line
    ctrl+enter = open alongside
    cmd+b = open/close side panel
    cmd+option+shift+> = move tab right
    cmd+option+shift+< = move tab left
    cmd+option+F = toggle fullscreen
    cmd+shift+l = highlight all selected
    cmd+shift+o = type fn name to jump to in open file
    cmd+option+shift+c = copy path of active filename
    
    Asana:
    cmd+shift+M: monospace highlighted text

👤 meribold
Mine is a file named oneliners.bash [1], and I have a Bash keybind (Control+I) [2] that runs fzf with that file as input and lets me select a command.

[1]: https://github.com/meribold/dotfiles/blob/e4dc5ee09d9e656346...

[2]: https://github.com/meribold/dotfiles/blob/e4dc5ee09d9e656346...


👤 Toreno96
https://github.com/Toreno96/dotfiles/blob/master/commands.tx...

Beware: I started this long before I became a fan of markdown, so the syntax of the whole file could seem as pretty weird, but it works for me. I mainly do `rg ~/commands.txt` in shell or `/` in vim when I need to find something in there.

There are also definitely too much commands dumped into the `Etc` section. I should reorganize this some day.


👤 rjh29
Try using flashcards (Anki) to remember shortcuts if you haven't yet. Just like with learning a foreign language, it's a quick way to get "action -> shortcut" mappings into your brain permanently for recall at any time, rather than relying on a cheatsheet.


👤 yourcousinbilly
I have a massive Cheatsheet for most languages and many packages- https://github.com/jsfuentes/Code-Cheatsheet

Also hosted on gitbooks: https://openai.gitbook.io/code-cheatsheets/



👤 kdeldycke
All my CLI: https://kevin.deldycke.com/2006/12/all-my-command-lines/

The one I'm probably the most proud of is that one-liner to produce GIFs:

  $ ffmpeg -i ./source.mp4 -ss 00:24:52.4 -to 00:24:57.0 -filter_complex "[0:v] fps=12,scale=width=480:height=-1:flags=lanczos,subtitles=source.mp4:si=0:force_style='FontName=Arial Black,Bold=-1,FontSize=26',split [a][b];[a] palettegen=64 [p];[b][p] paletteuse" ./meme.gif
Source: https://kevin.deldycke.com/2006/11/video-commands/

👤 darekkay
For bash, this is my cheatsheet (can be viewed from within bash): https://github.com/darekkay/config-files/blob/master/bash/.c...

I also maintain cheatsheets for some dev stuff: https://github.com/darekkay/config-files/tree/master/cheat-s...

And then there's my Flexbox cheatsheet: https://darekkay.com/flexbox-cheatsheet/


👤 vince14
Creating clips from tv recordings:

    ffmpeg -ss 01:59:00.000 -i "interlaced.ts" -ss 00:00:12.000 -t 26 -max_muxing_queue_size 1024 -c:a libopus -b:a 96k -c:v libx264 -crf 20 -vf "yadif=1" -profile:v baseline -level 3.0 -pix_fmt yuv420p -movflags +faststart -y clip.mp4
Make clip compatible with WhatsApp:

    ffmpeg -i in.mp4 -map 0:v:0 -map 0:a:0 -map_metadata -1 -map_chapters -1 -c:v libx264 -preset slow -tune film -crf 32 -c:a aac -b:a 128k -profile:v baseline -level 3.0 -pix_fmt yuv420p -movflags +faststart out.mp4
Copying in various terminals:

    PuTTY: select
    tmux: Shift + select
    cmd: select + Enter in quick edit mode
    Windows Terminal: select + Right Click

👤 moreati
I use twitter https://twitter.com/search?q=from:moreati%20ansible%20tip&sr.... More generally I have a Firefox bookmark set up so "tm search term" searches my previous tweets.

If I ever get round to editting/refining these I plan to do it on https://devhints.io - https://devhints.io/xpath and https://devhints.io/bash are so nice.


👤 bajsejohannes
My extremely short shell cheatsheet for things I do rarely enough to forget each time:

    Process substitution:
        bash: diff <(echo hello) <(echo world)
        fish: diff (echo hello | psub) (echo world | psub)
    
    Rsync directories:
        rsync -a -v --dry-run other:dir/ dir/
        Syncs the contents of dir (`/` needed)
    
    Script exit handler:
    bash:
        function finish {
          # exit handling code
        }
        trap finish EXIT # EXIT is special and works on normal exit as well as interrupts

👤 theginger
tab tab tab tab tab tab tab tab As someone who is quite severely dyslexic tab complete changed my life.

👤 ebfe1
As for me, I put together some spaghetti bash functions for taking note while using the terminal (eg run tnote function will let me select one of the last 10 commands, type a description of what it does and move on with my day...i can come back later and sort it out into my notes)...

I then write another small bash script that use consolemd and surge(probably will move to github page at some point) to generate a simple webpage with simple markdown JavaScript library to serve it up along with all the files generated by consolemd so i can use curl in terminal and have it colorfully displayed.

The cheatsheet site is here https://ch.ebfe.pw/.

And you can try it in terminal: curl https://ch.ebfe.pw/intel/splunk

And my code can be found here if you are interested:

https://github.com/santrancisco/cheat


👤 Helmut10001
I write down all my commands, ever issued in the terminal, in temporal ordered markdown files per task/project. This means I can go back, compare, see how I did it in the past and improve, if possible. Or investigate, why something isn't working (based on my noted commands, which is eaier in hindsight).

👤 chrismorgan
I keep some as shell aliases, shell scripts or Git aliases, but mostly I just use my shell history, finding commands with Ctrl+R and a few characters that I know to be in there somewhere. (If you want to do this, make sure you set its limit high enough so you don’t lose stuff.)

👤 ww520

    - List dirs and file count, to find dir with high file count.
      for i in /home/*; do echo $i; find $i | wc -l; done
      for i in /*; do echo $i; find $i | wc -l; done
    - Report space usage of the top level subdirectories.
      du -h --max-depth=2
    - Report space summary only, no directory output.
      du -s -h
    - Find out which device a directory is in.
      df /tmp
      df /home
    - Print of tree of directories
      tree .
    - Disk
      fdisk -l,  list disk partitions
      lsblk,  show a tree of disks, partitions, and lvm volumes
      lsblk -d,  show physical devices
      lsblk -io KNAME,TYPE,SIZE,MODEL,  name/type/size/model


👤 mehdix
Like some others, I don't have a list of cheatsheets except my bash history and my browser bookmarks. My bash history atm contains 4575 unique entries (I reset it a few weeks ago). It is like L1 cache to me for accessing most frequently used commands. Beside that I press `*` then Space in browser bar to search my previously bookmarks (I add keywords to them upon bookmarking by modifying the title). Beside these two, nowadays I write Makefiles in every project I work on. That way one can consolidate all commands that otherwise would be forgotten or scattered in configs, in one place.

👤 lynndotpy
I use these two commands regularly:

1. `history | grep somecommand` to see a command I used previously. (I have my history set to a bonkers high number.) 2. `curl cht.sh/somecommand` to see a crowdsourced cheatsheet for a command. See: https://cht.sh/grep 3. DuckDuckGo often has good cheat sheets, e.g. https://duckduckgo.com/?q=tmux+cheat

I also use fish, which has nice history-based autocomplete


👤 dolfins
I use navi: https://github.com/denisidoro/navi

> navi allows you to browse through cheatsheets (that you may write yourself or download from maintainers) and execute commands. Suggested values for arguments are dynamically displayed in a list.

It also lets you use cheatsheets from tldr and cheat.sh from it (besides your own cheatsheets). I've started to mostly just maintain my own, copying what I find most useful from those repositories, though.


👤 kbouck
For those with larger collections (I'm up to ~350 individual sheets), I'm interested to know you access (find/read/edit) the cheatsheets. My way:

(Mac Desktop): I use xbar [0] with a custom script to create a menubar nested dropdown of all of my cheatsheets. Selecting opens in typora [0] for reading/editing markdown.

(CLI): I use cheat [0] which consumes the same markdown cheatsheet content

    [0]: https://xbarapp.com
    [1]: https://typora.io
    [2]: https://github.com/cheat/cheat

👤 ozim
I keep my "lab" diary in txt files for each day and use VSCode.

Would not ever post any of it because these are mostly specific command lines.

Then with VSCode if I need to remind myself of something I search for command I might use and get my whole history with contexts. Then I copy paste/change and keep all that history there.

Never ever I will spend time organizing it into cheat-sheet because it will be waste of time anything that I put into cheat-sheet that I won't use in next 6-12 months will be just cluttering space on such cheat-sheet.


👤 tomcam
Go examples https://github.com/tomcam/goexamples

The least you need to know about GitHub pages https://tomcam.github.io/least-github-pages/

Soup to nuts psql tutorial https://tomcam.github.io/postgres/


👤 ripley12
On a related note: has anyone used a good tool for using/inserting these kinds of custom snippets? It would be neat to have a snippet manager with quick search bound to a keypress.

👤 News-Dog



👤 theden
Usually I'll just search online and read man pages (and experiment different options and solutions) but here's a short list of my most used aliases and functions, I guess that's the closest thing I have to a cheatsheet

  alias private='shopt -uo history'

  alias unprivate='shopt -so history'

  alias gl="git log --all --decorate --oneline --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an>%Creset'"

  alias gitlog='git log --graph --pretty=format:"%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an>%Creset" --abbrev-commit --date=relative --branches'

  alias sha256sum="shasum -a 256"

  alias sha512sum="shasum -a 512"

  alias flushDNSMac="sudo killall -HUP mDNSResponder"

  alias myip="dig +short myip.opendns.com @resolver1.opendns.com"

  alias m1="arch -arm64"

  alias x86="arch -x86_64"

  alias ibrew='arch -x86_64 /usr/local/bin/brew'

  alias upgrade="ibrew upgrade && m1 brew upgrade && mas upgrade"

  alias docker-clean='docker ps -aq | xargs -P $(nproc) -n1 docker rm -f ; docker rmi -f $(docker images --filter "dangling=true" -q --no-trunc)'

  alias autoscalerstatus="kubectl describe -n kube-system configmap cluster-autoscaler-status"

  alias evictedpods="kubectl get pods --all-namespaces --field-selector=status.phase=Failed"

  2qr() {
    qrencode "$1" -t ANSI256 -o -
  }

  curl_status() {
    if [ -n "${1}" ]; then
      curl -L -o /dev/null --silent --head --write-out '%{http_code}\n' "$1"
    fi
  }

  webp_convert() {
    local file="$1"
    cwebp -q 100 "$file" -o "${file%.\*}.webp"
  }

To minify everything (using https://github.com/tdewolff/minify)

  find ./dist/ -type f \( \
    -name "*.html" \
    -o -name '*.js' \
    -o -name '*.css' \
    -o -name '*.svg' \
    -o -name "*.xml" \
    -o -name "*.json" \
    -o -name "*.htm" \
    \) \
    -and ! -name "*.min\*" -print0 |
    xargs -0 -n1 -P4 -I '{}' sh -c 'minify -o "{}" "{}"'

👤 Dzugaru
Me and my friends have a funny habit calling these Spellbooks and pretending to be magicians. This leads to all sorts of hilarious interactions.



👤 arunsrin
https://notes.arunsr.in/

cheat sheets, notes, learnings all go here.


👤 swframe2
Sounds like we need a "github copilot" for the shell. bash on linux can already tell you which command you need to install if necessary. I really don't want to keep cheatsheets any more. I just want to write a shell comment and get the suggestions of the commands. It shouldn't be too hard I think. I guess the problem is how to pay for it.

👤 pppppppppps
Here is tool that I use very often: https://github.com/cheat/cheat

Super simple!

Here is example:

> cheat zip

# To create zip file:

zip archive.zip file1 directory/

# To create zip file from directory:

zip -r archive.zip directory/

# To create zip file with password:

zip -P password archive.zip file1

# To join a split zip file (.zip, .z01, .z02, etc):

zip -FF splitZipfile.zip --out joined.zip

# To list, test and extract zip archives, see unzip:

cheat unzip


👤 frefr
I have maybe 50 pages of cheat codes worth. I haven't had the time to post them somewhere yet. Remind me... a week from now :D

👤 pmoriarty

  * Some zsh tips
  ** Zero-pad (preserve zeroes) a variable
  Either
  *** typeset -Z2 FOO
  *** printf %02 $FOO
  ** Do arithmetic
  *** examples
  **** addition
  (( count = $count + 1 ))
  ** Loop through a directory of long filenames with spaces
  while read -r line
  do
    echo "'$line'"
  done < <(print -l *(/))
  ** Shell return codes for piped commands
  echo foo | grep bar | tr z a | cat
  echo ${PIPESTATUS[@]}
  
  0 1 0 0
  ** Pass file contents as an argument
  *** foo "$(

👤 aaronax
I actually do most of my tasks in various GUIs. This has a large advantage in discoverability (and for my brain: memorability). So I find no need for cheat sheets in general.

If on command line, I am typically following an installation guide or something so no need for a cheat sheet.

If scripting, I refer to past scripts or use a search engine.


👤 xupybd

👤 sorokine
I keep a long cheatsheet for many-many-many years as a github gist: https://gist.github.com/sorokine/7b032ccb39514e2a64801f63d9e... (yes, it depends upon my dotfiles)

👤 hoseja
I keep a printed copy of C++ operator precedence around the desk.

https://en.cppreference.com/mwiki/index.php?title=cpp/langua...


👤 pplonski86

👤 subpixel
My team has a shared knowledge base CLI set up via kb: https://github.com/gnebbia/kb

It’s like a group-maintained collection of cheatsheets. The git sync is buggy but I’ve made it work.


👤 Dachande663
I feel almost embarassed for sharing some of these, but I either remember things the first time I'm exposed to them or am forever doomed to have to look them up.

    ln -s /the/target the_link (can never remember order)

    for f in *.mp4; do mv "$f" "$(echo "$f" | sed s/S1E/S01E/)"; done

    tar -theuniverse (insert xkcd here)

    grep -E 'foo|bar' file (can never remember the flag for regex)


👤 tr1ll10nb1ll
My cheatsheet happens to be just my muscle memory. And, for applications I don't use often, mostly just intuitive guesses.

Sounds pretty messy but I've never felt the need to maintain a cheatsheet.

I don't see why one would. I know I'm possibly being naive.


👤 remram
I just use my bach history. I set it to never expire and I back it up, so it has years of commands I can search, either with Ctrl-R or history|grep. Sometimes I put comments in them to help with searching later.

[edit: turns out a lot of people do that!]


👤 psyc
I tried to maintain one many years ago, but I could never get in the habit of being consistent about it. So it was short-lived and didn't get much use. I do refer directly to past code all the time, and that serves me fine.

👤 sdwolfz
The whole purpose of this section in my website is to have sharable cheat-sheets that I own: https://www.codrut.pro/snippets/

👤 tailspin2019
Weirdly, I’ve never thought of keeping a single cheatsheet for command line snippets.

I’ll usually put relevant commands into my project README files but your post has inspired me to start a global cheatsheet of common commands too!



👤 sharjeelsayed
Here is mine Curated Self Study Guide for Computer Science, DevOps, SRE & SysAdmin (https://Learn.SharjeelSayed.COM)

👤 INTPenis
I never did have a cheatsheet. Strange since I've been on the *nix journey since 1998.

I do like the cheat command but it's still not habitual to use it, I grew up with Google so I'm far too dependent on it.


👤 dpz
Don't really use a cheat sheet. history and fzf is mostly what i use

👤 ashwinne

👤 jwilk
POSIX sh parameter expansion cheat sheet:

https://github.com/jwilk/sh-param-expansion


👤 habibur
All kept in batch files in a directory with a meaningful name. When I need something what I do a ls on the directory and can recon the command I need.

👤 donbrae
I use my GitHub gists as a cheatsheet: https://gist.github.com/donbrae.

👤 blooalien

👤 hackersonggg
A friend of mine made this awesome collection: https://devhints.io/

👤 memorable
I don't really use a cheatsheet, but I stored all of my snippets in my shell configuration file. In this case, I use fish shell.

👤 warrenm
my only cheatsheets are customer-specific ... so they're basically unshareable (the general ideas of what's being done apply all over, but there's a lot of environment-specific stuff that I don't want to have to remember all the time :)

👤 valbaca
less "cheatsheet" and more just general notes for languages and vim and such:

https://github.com/valbaca/notes


👤 nindalf
I used to keep a list of snippets, but I found a tool that did it better - `tldr` (https://tldr.sh). I can use this tool from the command line instead of going searching my notes for a snippet.

`tldr tar` or whatever.


👤 acutesoftware
### Disk and Hardware

Hardware Summary

    sudo lshw -short
    
Get the total disk space left and summary of folder usage

    df -h .; du -sh -- * | sort -hr
    
Simple partition summary

    lsblk

What version of Linux are you running

    uname -a
    > Linux TREEBEARD 4.4.0-98-generic #121-Ubuntu SMP Tue Oct 10 14:24:03 UTC 2017 
      x86_64 x86_64 x86_64 GNU/Linux
    
    lsb_release -a
     > Distributor ID: Ubuntu
     > Description:     Ubuntu 16.04.4 LTS
     > Release:         16.04
     > Codename:     xenial
 
 What flavour of Ubuntu are you running (see https://itsfoss.com/which-ubuntu-install/ )
 
     cat /var/log/installer/media-info
     > Unity
   
How long has the PC been running

     uptime
     >  23:09:26 up 61 days,  8:28,  1 user,  load average: 0.82, 0.48, 0.34

Count files in folder and sub folders

    find . -type f | wc -l

### Files and Folders

Get a tree view of subfolders

    ls -R | grep ":$" | sed -e 's/:$//' -e 's/[^-][^\/]*\//--/g' -e 's/^/   /' -e 's/-/|/'
Find the most recently used files including all subdirectories

    find . -type f -mtime -7 -print0 | xargs -0 ls -lt | head
  
Find string in all files (example searchs logs for Exception)

    find /var/log/www.lifepim.com -type f -print0 2>/dev/null | xargs -0 grep --color=AUTO -Hn 'Except' 2>/dev/null
Find a string 'blah' in all files with recursive (deep) search from current folder '.'

    grep -Rnw '.' -e 'blah'
Limit above search to only .html files

    grep -Rn --include=*.html '.' -e 'blah'
    
    
    
### Processes

List all processes

    ps -ef
   
   
Show a tree of processes

    pstree
   
   
Find the processes I am running

    ps -u duncan
    
    
Get list and PID of specific processes (eg python)

    pgrep -a python
    
Show all processes and usage

    top
    htop   (will need to run sudo apt install htop first)
    
### Network

Get IP Address and network details

    /sbin/ifconfig
See list of PC's on the network

    arp -n   
    
    ip -r neigh
    
    nmap -sA 192.168.1.0/24
    
Lookup name of IP Address

    nslookup 162.213.1.246
    > Non-authoritative answer:
    > 246.1.213.162.in-addr.arpa      name = wssa.beyondsecurity.com.
  
Get the IP address of a domain name

     host www.acutesoftware.com.au
    > acutesoftware.com.au has address 129.121.30.188
  
  
 Show the routing table

    route 

Port scanning

    nmap

 
 ### Shell tips
 
 Show top commands from your shell history 
 
     history | awk '{a[$2]++}END{for(i in a){print a[i] " " i}}' | sort -rn | head
          35 cat
          25 awk
          18 pwd
          15 ls
          14 cd
     
     
  
### Data Collection

Download a file

    wget http://www.acutesoftware.com.au/aikif/AIKIF-Overview.jpg
    
Download a site for offline reading

    wget --recursive  --page-requisites http://www.acutesoftware.com.au/cont_articles.html

### Data extraction

Get a list of URLs from a html file (like an exported list of Chrome bookmarks)

    grep -Eoi ']+>' source.html | grep -Eo 'HREF="[^\"]+"' | grep -Eo '(http|https)://[^/"]+' > urls.csv
    
Grep log files

    cat /var/log/www.lifepim.com.access.log  | grep "POST"          # number of posts to lifepim
    cat /var/log/www.lifepim.com.access.log  | grep "login" | wc -l # number of login page accesses
    awk '{print $1}' /var/log/www.lifepim.com.access.log | sort | uniq -c   # count per IP address
    awk '{print $7}' /var/log/www.lifepim.com.access.log | uniq  # list of pages accessed
    
    cat /var/log/www.lifepim.com.error.log | grep "Exception" | uniq      # list of exceptions
    
    awk '{print $11}' /var/log/www.lifepim.com.access.log | sort | uniq -c | grep -v "lifepim"  # count by referrers
    
        1 "https://newsbout.com/id/19184625381"
        1 "https://umumble.com/links/156005/what-software-will-you-trust-when-you-get-senile%3F"
        1 "https://www.producthunt.com/ask/616-what-s-the-best-personal-knowledge-base"
Looping through list of gz files and grepping for blog hit count

        for i in /var/log/www.lifepim.com.access*.gz
        do
            echo -n "Checking zipped logfile $i - "
            zgrep '/blog/' "$i" | wc -l
        done


        Checking zipped logfile /var/log/www.lifepim.com.access.log.2.gz - 45
        Checking zipped logfile /var/log/www.lifepim.com.access.log.3.gz - 112
        Checking zipped logfile /var/log/www.lifepim.com.access.log.4.gz - 92
        Checking zipped logfile /var/log/www.lifepim.com.access.log.5.gz - 62
        Checking zipped logfile /var/log/www.lifepim.com.access.log.6.gz - 64
        Checking zipped logfile /var/log/www.lifepim.com.access.log.7.gz - 85
        Checking zipped logfile /var/log/www.lifepim.com.access.log.8.gz - 213
        Checking zipped logfile /var/log/www.lifepim.com.access.log.9.gz - 80
### Date and Time

Display Annual Calendar for current year

    cal -y 
   
   
Show the current date in ISO format ( yyyy-mm-dd )

    echo $(date -I)
    
Store the current date / time as string in a bash variable

    DATE=`date '+%Y-%m-%d %H:%M:%S'`
    echo $DATE
    
   
### SQL tips

Show table size of selected tables in a schema

    SELECT table_name as 'Database Name', 
    sum( data_length + index_length ) as 'Size in Bytes', 
    round((sum(data_length + index_length) / 1024 / 1024), 4) as 'Size in MB' 
    FROM information_schema.TABLES where table_name like 'as_%' or table_name like 'sys_%' 
    GROUP BY table_name; 
Get a list of column names for a table

    select * from information_schema.columns where table_name = 'as_task';
Show usage in log file grouped by date

    select DATE_FORMAT(log_date, '%Y-%m'), count(*) from sys_log group by DATE_FORMAT(log_date, '%Y-%m') order by 1;
Show usage by user_id and date

    select log_date, user_id, count(*) from sys_log group by log_date, user_id order by log_date;
Show users by week

    select WEEK(log_date), max(log_date) as date_until, count(*) as num_user_actions, 
     count(distinct user_id) as active_users_this_week from sys_log 
     where DATE_FORMAT(log_date, '%Y-%m') > '2018-05-05' group by WEEK(log_date) order by 2;

👤 jiggawatts
pwsh

👤 nathias
tldr