If you already know all the stuff which I write in the subtitle, please play around with the following packages
1. Operation System (OS)
Operation System Wiki We are using different operating systems all days, now just focus on your computer:
- Windows (Our laptop, you need to pay)
- Unix and Unix like operation system
FAQs of OS
What’s your point to say this?
We need an app/software, which is called Bash, installed into your computer.
Why we need this?
- We can use ssh via Bash connect to our lunchbox server.
- We can use Bash easily run GUIDE.
- We can use other powerful GNU software. eg. grep, sed, awk
Why do we need to use grep, sed, awk? Can we just use excel, modified data by hand?
- Yes you can, IN SMALL DATA SET!
- One question Valid Number Leetcode, if you have 1,000,000 recorders.
- I may teach in the later semester.
- Do I need to download/install Linux? Do I need to buy a Mac?
No! You don’t need to! Our server is free, you only need a software and connect to the server(lunchbox), then you can try everything! Also, Windows 10 has the bash. Don’t worry I will teach you some basic stuff.
- What should I do?
We only need Bash to manager our computer, also can connect to the server. That’s all, the following steps you can do to say whether you can get the bash in your computer
open command prompt; type `bash` If you have a problem with bash: if you are using Windows 10: see [Windows 10 Installation Guide](https://msdn.microsoft.com/en-us/commandline/wsl/about) else if you are using Windows 7: try install [UnxUtils](https://sourceforge.net/projects/unxutils/)
- Do I need to download/install Linux? Do I need to buy a new computer and install Linux?
No! Linux is another type of MacOS, although there is some difference between MacOS and Linux, but you will hard to tell the difference. We are not computer science engineer. Don’t buy a new laptop and install Linux…
- What should I do?
- Mac already have the terminal.app, you can use spotlight to find it. Bash already running inside.
- [Optional] Mac also has another fancy terminal app call iterm2 I used it.
I think you should have the ability to figure out any questions by yourself. Please use StackOverflow wisely.
2. Bash 101
I hope you already have bash in your hand. Here is a great course
summary (Thanks to Professor Cecile Ane):
- directory structure, the root is
- relative versus absolute paths
~your home directory
- tab completion to get program and file names
- up/down arrows and
!to repeat commands
||who am I? to get your username|
||print working directory. where am I?|
||list. many options, e.g.
||remove (delete) directory, if empty|
||move (and rename). can overwrite existing files, unless
||copy. would also overwrite existing files|
||word count: lines, words, characters.
||because “less is more”.
||first 10 lines.
||last 10 lines.
||filters out repeated lines (consecutive).
||cut and return column(s).
||shows the history of all previous commands, numbered|
Things you can do
Question: I want to execute my guide anywhere, not by copy my guide to anywhere.
We all use hyperlink every day, we like to create a desktop icon in windows or drag the program into your mac Dock, so that we can easily execute the program. Why we can do it easily because we know where is the program. How to let our computer know where is
guideor in the future, if you need install some software, how to let our computer know.
Bash has a variable call PATH, which specifying a set of directories where executable programs are located. So that when you type
guideinto terminal or command line, bash will search that directory, if there is a program call
guide, it will execute it, if not, they will say
command not found
So that we need to tell our computer “next time you need to search an additional directory where guide is located”
The following steps help you do this (Only do it in the first time), and you can execute guide anywhere on your own computer after.
- Create the following directory on your computer
~/local/bininto your environment PATH variable
mkdir -p ~/local/bin echo 'export PATH=$PATH:~/local/bin' >> .bash_profile source ~/.bash_profile
mkdir -p flag: no error if existing, make parent directories as needed
You can also use
man mkdir in bash to see the details
Windows or Linux
If you use bash in Windows command prompt, you need to download Linux version of guide.
Download or move your
guide program not zip file into
cd ~/local/bin wget http://www.stat.wisc.edu/%7Eloh/treeprogs/guide/linux/64bit/Intel/guide.gz gunzip guide.gz chmod a+x guide guide
If you want a different version of guide, change the URL.
cd ~/local/bin curl http://www.stat.wisc.edu/%7Eloh/treeprogs/guide/mac/absoft/guide.gz --output guide.gz gunzip guide.gz chmod a+x guide guide
Go to any directory from bash via
cd in your computer, type
guide see what’s happening.
Server (Your friend’s computer)
- What is lunchbox?
lunchbox is our stat department server. Many of you should already have relative experience with server, just like AWS.
You can take it as your friend’s computer, and we can use your friend’s computer to run your own programs.
- Ok, why I need this?
You can definitely ignore it leave if you refuse to learn!
- When I need this?
When you need your friend’s computer. For me, I usually need to run some simulations thousands of times, some optimization jobs take weeks. So that I can throw everything into the server.
- But I need my computer to watch a youtube video.
- I don’t want to save lots of files into my own computer.
- I want to play computer game when my simulation is running.
- I just want to save my electronic bill…
- How to use lunchbox?
Change username to your own username! The password will not show anything, but you can keep typing, and use
return to execute.
Since it is your friend’s computer, in the beginning, it doesn’t have anything. Even don’t know what’s your homework.
Your friend: Ok, you can use my computer only for homework and computing, you can not play games or watch youtube, promise?
You: Sure, what do you have?
Your friend: I only have a bash terminal…haha, no chrome:P
You: …Ok… Do you have
Your friend: YES!!! Type following in bash
Your friend: You classmates also want to use my computer, so I want to save all your file into one place, OK?
You need to create a directory under /workspace, username is your username to login lunchbox
mkdir -p /workspace/username cd /workspace/username guide
Your friend: Let’s try R? DON’T JUST TYPE R!!!!! It will make my computer too slow.
srun R --no-save
You: Hope can I upload my homework?
Your friend: You can install SecureFX, but today let’s use another way to upload your file
Open a new Command Prompt or terminal. You new terminal is not connected to your friend’s computer yet, you can use
lsto see everything on your computer not the server.
Your current terminal A already connected to your friend’s computer, then open terminal B.
- Open a new terminal B, or command prompt
- Upload your file via terminal B use command
-r, which means directory, to lunchbox
- Check your file from terminal A
More details about
I hope you can find your homework easily by this way. Suppose your homework is in the directory
bash cd /mnt/c/Users/WinUserName/Desktop ls # you should see stat479 scp -r stat479 email@example.com:/workspace/username
Drag your homework directory into your terminal, it will show where is your homework directory.
Suppose your homework is on your Desktop directory
cd ~/Desktop scp -r stat479 firstname.lastname@example.org:/workspace/username
lunchbox 102 submit job
You can follow the Professor Loh’s slide to create a submit file for guide_impute.r, I will just show a simple example to write “hello world” use R.
Still in terminal A.