The problem is, I know so little that I can’t actually do anything with this knowledge. I suppose I’m looking for a tutorial that will teach me to be comfortable with the command line and a Unix environment, while also teaching me to code a language. Where should I start?
My strategy to solidify my linux skills was simple: commit to linux desktop. We spend most of our time on our desktop, may as well let that time earn you skill points. I picked ubuntu and use the same on servers. You naturally learn a bit every day through daily use. This really adds up over time without extra effort of using a side system. Getting good at windows or osx is a waste of your time unless you are specifically into those platforms.
As for a language, pick one that is (1) easy to start with (2) covers advanced use cases, (3) sound in its design, (4) general purpose, (5) concise. That rules out C (due to 1), JS (due to 3 and 4), php (2,3,4) and most things really.
For me that left me with ubuntu and python, I never looked back. (Yes I know many other things but that is my home base.)
[1]: https://missing.csail.mit.edu/
[2]: https://software-carpentry.org/lessons/index.html
[3]: https://mywiki.wooledge.org/ParsingLs
1) install the language of your choice and start a simple web site using the languages webserver. View the website in your browser on Windows. A Python Flask 'hello world' or a Node one?
2) Set up an SSH server and remote in from the Windows machine. Learn how to set it up to use keys only instead of passwords. Find out why that is a good idea
3) learn Git. Github has some nice tutorials.
4) set up NGinx to run your website above.
5) set up Nginx for https
6) set up LetsEncrypt.
7) set up a free AWS account, start a server and put your website on it using SSH and git.
8) look through the ssh log files to see login attempts, and the Nginx logs. Imagine there were millions of rows, how would you find a certain one, or the last few? (head, tail, grep, cat).
That should get you somewhere.!
- https://cmdchallenge.com/. This set of increasing-in-difficulty challenges should help with structure in terms of learning a bit more.
- https://explainshell.com/ can be helpful as well.
- Don't forget the man pages! `man - I haven't used it personally, but there is a lot of info at http://www.opsschool.org/ As for learning to program, I suggest a very good starter language is Python, and its built-in, first-party language reference/tutorial I found helpful when I learned it: https://docs.python.org/3.8/tutorial/index.html As for "what" to program after that, I don't have any great resources for what constitutes some good initial programs. Off the top of my head, here are some ideas about places to start which are doable with only inputting and outputting text: - FizzBuzz - Play Rock Paper Scissors with the computer - Play Tic Tac Toe with the Computer - Sieve of Eratosthenes - Conway's Game of Life - Given a piece of text, count how many occurrences of each word there are and display the top ten.
Start with installing a distro of your choice in a virtualization tool, I always recommend Ubuntu on VMWarePlayer because it was what I started with. Distro/Virtualization Software doesn't matter, what matters is that you use it.
Then I have students work through Linux Journey [1] one module at a time, while being available to answer questions as they go.
Once a student has finished the LJ modules I have them play the Bandit Over the Wire [2] game up to challenge 15.
After that I usually teach Hardware and Operating Systems, but if you are just looking for general ability to do things, skip to the CyberAces Linux modules. [3]
Once you have finished those, do the CyberAces Bash Scripting module and the CodeAcademy Python [4] course and you'll have enough scripting ability to be dangerous.
From there, just keep using Linux as your daily driver and you will continue to improve, especially if you take on programming projects.
All of this can be done by yourself, but I teach a free online course on Computing Fundamentals and Security if you are interested in it being more of a guided experience. [5]
[1] https://linuxjourney.com/ [2] http://overthewire.org/wargames/bandit/ [3] https://tutorials.cyberaces.org/tutorials.html [4] https://www.codecademy.com/learn/learn-python [5] https://www.hoppersroppers.org/course.html
Also check github for a bunch of repos that contain biolerplate code that is used in most deamons illustrating signal handling, forking, etc.[2]
Also I suggest taking some open source projects from Daniel Bernstein (DJB) as examples on how to write secure code. qmail, djbdns, daemontools ... there are lots of great ideas there[3]
Write a couple of simple programs that utilize your code and expand from there, learn the plumbing, e.g. write a Makefile, learn autotools/autoconf, how to write tests, how to use gdb, how to create shared libs and link a program LD_LIBRARY_PATH/ldconfig, etc ...
Most important think about something that you like to write and go from there. Back in the late 90ies I studied RFC's (MIME, SMTP, etc) and then implemented things that I could actually use myself. Here is a recent project I did some weeks ago (an extreme edge-case for security maximalists which will never be used by anyone other then myself, ... but I really enjoyed writing this and learned a bunch of new stuff while doing so)[4]
if you need ideas or help with looking at your code, don't hesitate and send me a mail.
[1] https://en.wikipedia.org/wiki/Advanced_Programming_in_the_Un...
Get familiar with the interface and the concept of a "terminal".
Then try do some simple stuff. Maybe set up nginx (or Apache) to run a web server that you can access via your host machine.
Thereafter try doing some Linux admin tasks, like creating users, switching users. Maybe even set up SSH and try accessing the virtual machine from your host machine via PuTTY.
After, maybe try spinning up a simple node web app on the virtual machine and access it from a browser on your host machine.
By this time, you'll probably get a sense of what you want to explore next! And remember, Google it's your friend for all of the above!
All the best :)
Keep the terminal open, if it can be done in a terminal it may turn out you prefer it when you get used to it. Torrents, IRC, playing music, email, file explorer, whatever makes you stay in there. It makes you comfortable navigating the system and you get to see what a GUI hides from you.
Programming? You're not gonna benefit from diving into any deep ends there, continue with JS if that's what you like. Learn databases and web servers, here your new linux skills comes in use. Try some stuff and see what you like.
My second piece of advice would be to learn C from this book: https://en.wikipedia.org/wiki/The_C_Programming_Language
After finishing it and completing all the exercises, you should be ready to go learn anything else you want to.
For Unix, install linux on your computer and use it daily. You’ll be forced to learn.
Here are some of my favorite books:
https://learning.oreilly.com/library/view/ubuntu-server-cook... - really good one for learning real uses of linux
https://learning.oreilly.com/library/view/raspberry-pi-cookb...
https://learning.oreilly.com/library/view/raspberry-pi-netwo...
https://learning.oreilly.com/library/view/exploring-raspberr...
https://learning.oreilly.com/library/view/raspberry-pi-hacks...
1) Get a good linux, you can start with ubuntu. After getting comfy with ubuntu try another os, debian,And eventually to the hardcore ones like Arch, Artix and others. Try to do your usual thing with commandline. For example, How to copy using terminal,How to open a file with terminal, How to get lots of terminals inside terminal. Stuffs like this (trick is simply go ahead and add "How to do it in terminal" ;) ) Google them up. (www.giyf.com) Try to rice your system. It teaches crucial aspects about the appearance,usability etc.
2) Get a good book. No starch press has some good ones, even O reilly gets you into shape. The trick to good book is simple: Get a book, read it. If you love it finish it. There are open source notes as well. Awesome cli github is similar.
3) Try to create your own tools using 1,2 this will allow you to bend your brain. Stuffs like How can I get hackernews comments in my terminal, How can I create a shell script to remind me that 20mins have passed (pomodoro) and stuffs like that.
I have some starred repositories and Some simple tools of my own (Aeres-u99 @github) to check for some example.
I think with this (depending how extensively you do them) you can be decent
Read and lightly work through The C Programming Language (K&R). Code along with the book, don’t feel you have to really deep-dive anything.
You don’t have to learn C well at this point. Just learn from it.
Then start reading The Art Of Unix Programming. Do this casually. Read a bit, code a bit. Between K&R and TAOUP, you’ll be inspired by to write at least two small projects.
To supplement K&R and TAOUP, begin reading Unix Network Programming.
If you have access to a Mac or a Linux machine, explore the man pages.
If you’re on a Mac, explore the man pages __and__ the code that lives on your system.
One way to do this is to use something like vscode’s intellisense to “Go to Definition.” E.g., when you’re coding an example from K&R, right click on one of the included headers, and click “Go to definition.” It will take you to the header file. Look through the code. Then you could look at the code from FreeBSD or OpenBSD on GitHub.
Edit: Just wanted to add, by doing the above you’ll get exposure to your system in new ways. Moreover, you’ll find yourself learning Unix out of necessity, for fun, and by accident. You can practice the art of Unix programming in almost any language at this point. C is unix-y, but so is Rust.
- Customize your bash prompt, create aliases for some common commans (ls, df are good)
- Build a LAMP stack, compile everything from source tarballs. Get it working so that you're able to browse your webserver from your phone.
- Update a sound or video driver
- After you're comfortable, recompile the kernel
- Through these, do as much in the command line as you can. Get comfortable with mv rm chmod chown and so on.
Any time you have edit a text file, use vim, nothing else.
Despite the title, it's not that advanced; the "advanced" here refers to system-level programming, not application.
Note, for this you probably need to learn C.
K&R ed2 remains an excellent book for this.
Honorary mention - Kernighan and Pike - Unix Programming Environment. But it's a little dated.
Above all, do. Get a Ubuntu or Centos image, install developer tools (gcc), manpages, and write some code.
For regular expressions, watching a small video which explains how they work is definitely more productive, but if you could find some time to spare , try checking how compilers are made, / what constitutes a language (language theory), and automata, it definitely has some math in it , but it s worth it, it gives you an extra mental image , which helps you get a better understanding of where you stand and what you can do. Since apparently regex is also a language, which helps you describe chunks of an other languages and it also has some limitations to be aware of.
I have delayed making the switch because I have lot of things to learn (currently relearning math, testing out different deep learning architectures, and learning JavaScript (second language after Python)) and not sure if should prioritize learning it.
I still need to get better with GitHub which is something I clearly see the benefits of. Also need to get better with a text editor , using Visual Studio Code , mainly just for text completion in Python and JavaScript, so need to learn the additional benefits. Also want to test sublime text.
I see recommendations for vim, but I wonder if that will make me too twitchy and will start to type before thinking.
So due to the amount of things I am trying to learn, I have delayed learning Ubuntu.
I know Windows 10 now has built-in bash capabilities but I wonder if there are other benefits I am missing with Linux and worth learning.
I was learning C programming at the time, so beyond dealing with bizarre things like my program crashing due to "segmentation fault"s, I was trying to understand what was going on. The first time I opened vi, I couldn't figure out how to get out. Once I did, I used `:sav %` followed by `:q` for longer than I care to admit.
If I had to recommend anything to someone just learning, I'd recommend learning these commands:
cd ls touch rm # but be careful! ps head tail nano who whoami man apropos # this is technically man less
These will help you do a lot of what you need to do on Linux about as well as you can do similar things on Windows. Once you've mastered those commands, learn about piping `|`, grouping `{ foo; bar; }` and sub-shells `$(foo)`. Learning about variables `$foo`
Then read up on:
grep sed cut find awk # really just `awk '{ print $2 }'` xargs env export test # [ foo ] source # . foo crontab top # or even better `htop` shell control flow
When you're done with that, learn
vim # `vimtutor` can help emacs
After that, there's tons of other tools you'll pick up as needed:
dd df du fsck and a bunch of other system-related tools
You'll learn a ton as you move along.
Just remember that for the most part, each tool solves approximately one problem, and the complex behavior comes from chaining tools together. Chaining mostly involves `|`. Once you see that at work, you probably won't want to go back to the crutch of point-click-drag.
Best of luck!
Since computing was my escape from a pretty unhealthy environment at home (alcoholics, TV) and computer games/hardware were prohibitively expensive beyond Quake 2, routine saved the day: go to high school, come home and zone into computer for 4 years. A couple years in I started getting small gigs, then a lot of them, then got on the FTE track and hit $200k/yr by 30.
As an adult, I’ve had 6 years of fitness starting with forcing myself to spend 45min/day with no plan in the gym until it felt weird to not go, and now I could easily get on track to become a trainer if I wanted. I got 50lbs overweight over a few years from being so steeped in building software and networking with beer and pizza. I’m mistaken for early 20’s at age 35 universally.
You have to intrinsically enjoy it, and then make time for it.
Some may argue that you can get the information in this book for free, by consulting man pages and such. Sure you can, but it will just take you much longer. This book will save you time.
My personal suggestion would be to do both. Follow Scott Berkun - his blog, Twitter, he knows his stuff.
You also need to some sites and courses to guide you throughout your journey. This article (https://blog.codegiant.io/5-best-websites-for-learning-progr...) has some cool sites for learning programming.
All in all, the best way to excel is to find a mentor.
https://www.edx.org/course/unix-tools-data-software-and-prod...
More info about week 1: https://www.reddit.com/r/linux/comments/flmr8o/enjoying_the_...
It can be pretty overwhelming to use the terminal so you may have to some reading to get yourself really comfortable.
I usually go to books/(book pdfs) to help me start out because they have conceptual explanations about the syntax which really helps my understanding.
Maybe this one would be good? The Linux Command Line, 2nd Edition: A Complete Introduction
Once I'm comfortable based on the book, using StackOverflow and other websites becomes much, MUCH easier.
In the process, you'll learn BASH which is used for scripting.
The automatic driver detection when you install it can be handy, the Arch repositories are easier to use and include the same software you would find on any other machine, and the Arch wiki still applies, allowing you to find information on any part of the system you have questions about.
In addition, I think Arch derivatives in general encourage a healthy reliance on the command-line, which is actually what you want to learn how to use.
Keep projects small, just patch together open source code for fun, and read the open stuff you do use.
You could also switch the GUI environment of your raspberry pi. Install headless raspbian and install he GUI from the CLI. Just hack!
My koan is video poker, I’ve rewritten a whole game about 3 times. I also like writing prime number finders in new languages.
LFS is well-document and maintained, and levels you up in a jiffy on the practical basics.
It's not about programming per se, but covers quite a few concepts you're expected to "just know".
Plenty of pointers to more advanced concepts, too.
1) Do all day to day tasks with the CLI. It will force you to challenge yourself.
2) Learn line editing with GNU Readline. What is this? E.g., when you have a bash shell open and you hit Alt+B, your cursor moves back a word. This will make your time in the shell a lot more pleasant. Similarly, the basics of VIM, which will make text editing a lot more pleasant. Less friction means everything comes quicker, and you get tighter learning iteration loops. Don't go overboard on this: it's mostly about navigating around and editing text, not installing plugins or anything beyond an hour or two of study.
3) Don't waste time purposefully memorizing userland tools. You'll forget them in a week, and most of the time they're pointless clutter in your head. There is zero value in memorizing all the options for `tar`. Memory will come naturally with use.
4) Do not waste time building the perfect set of dot files (e.g. `.bashrc`, `.vimrc`). Know where they are, but futzing with them should be <1% of the time you spend learning.
5) Do not waste time building a Linux desktop, configuring a window manager, etc. Linux is, first and foremost, an operating system for servers, and the skills needed to get a polished Linux desktop user experience are entirely separate from (and usually dependent on) the skills needed to get a solid Linux server environment up and running.
6) Man pages are your friend. Before Googling how to do `something`, try `man something`. If you don't even know the man page you're looking for, try `man -k something`. If it's totally new, read the whole man page. If you're just looking for an obscure `tar` option, just skip to that section.
7) Cert book for RHCSA. Honestly pretty much every non-junior developer should have the skill level of a RHCSA. It teaches you how an entire system fits together. Note you don't have to pony up some obscene fee to RedHat to actually take the test and get the cert (though, as far as certs go, it's one of the better ones); the point is the content. And actually do all the labs/exercises in whatever book you get.
8) Advanced Programming in the Unix Environment (APUE). This is less critical and only relevant once you are vaguely at the RHCSA level (and know how to program), but it can give lots of valuable knowledge about the programming system interface.
2. Learn the stack that follows from 1, just barely, not too much theory. You want to start as k with 3.
3. Build something using the stack you learned. Start even if you are clueless of how to do it, and google every question you encounter. Stackoverflow will become your friend.
Repeat 2 and 3 for the rest of your life.
Next, get a book on bash or other shell. You’ll learn the most important parts quickly. Then do the same for Python.
If you make it that far, read the art of Unix programming to approach greybeard status.
Find someone experienced and ask them to mentor you. (Online of course, but use screen sharing and audio.)
Once you grasp how powerful this closed system of incrementally processing text is, you are sort of unstoppable at the command in a *nix system.
Pay careful attention to regular expressions, which are a universal tool in Unix for processing text. Along with Unix' treatment of files as a stream of newline terminated strings, this how Unix gets away with working with unstructured files instead of something like XML or S-expressions): that is, most Unix tools expect you to work with a file (or file descriptor, which is a pointer to any file-like thing in memory), and to process it one line at a time, and to use regular expressions to parse those lines into fields.
Working with lines using regular expressions is how the original editor, /bin/ed, was conceived, before the existence of visual editors like Vim and Emacs: you can find, print, and also edit lines in the file using regular expression syntax in ed, and this knowledge, btw, translates to fluent use of regular expressions in both sed ("stream editor", or basically a non-interactive version of /bin/ed, which essentially executes your editor commands and then exits), and egrep (which searches for lines matching your regular expression).
In fact, once you start to see these pieces come together, look for examples in Kernighan and Pike[0] which demonstrate the use of awk. Awk is an extremely powerful but also extremely simple tool for processing data in Unix, using all the concepts I just described, and not much else. It has a few extra features, like variables and loops, which you'd expect in a programming language, but you are expected to use Awk exactly like you'd use things like grep, ed, and the shell. It's just that now you've got this very compact, powerful notation for splitting records (lines) into fields (regular expression matches), and then doing arbitrary things with them to spit text back out. You could even write your own database using something like Awk, and you can read about how to do database-like tasks in the classic and accessible book on Awk by the original authors of the tool (Aho, Weinberger, and Kernighan, whose last initials together form the name AWK)[1].
Then, much later in life, when you find yourself doing some fancy stuff in Haskell and creating a DSL to do things like process pipelines, you realize that Awk was actually way ahead of its time.
[0] https://en.wikipedia.org/wiki/The_Unix_Programming_Environme...
[1] https://en.wikipedia.org/wiki/The_AWK_Programming_Language