Topics Map > Computing > linux
Topics Map > Computing > research
Specific Commands to use with Research Computing
This is a page where specific commands that are used when doing research are discussed.
Many of the above commands are run from the command line, and are used to help you understand how the server will process your request. Often, they are run with a switch, usually a dash with a letter, to help modify the output of the command. All of the commands listed below are run with certain command line switches. However, if you type man renice, for example, you could learn about ALL of the command line switches available for the command renice. This is a good habit to start to learn more about how commands work.
nice and renice puts a flag on the script you are running. It is a number between -20 and 19. By default, your scripts run at 0. However, if you wanted to be nice, you could specify your scripts with a specific nice level. This means your scripts would gracefully step aside for other higher priority items. -20 is the highest priority. 19 is the lowest priority.
When you nice a script, you are saying, I'm sharing this server, and I'm happy to step aside if you need to use this server. The CPU will continue processing my scripts after your scripts are done.
nice -n 10 myscript
This sets the myscript to 10, which makes it happen after other more important processes.
Renice allows you to reconfigure an already running program to a lower priority. However, programs that already running have to be referred to by the PID number.
If you type top, you'll see a list of all of the processes running on that server at any given time. You should be able to see what your PID is.
So, let's say you figure out your process is 21827.
renice 10 -p 21827
would allow you to reset your nice level to 10 on that process. The process would continue to function, just becomes more flexible with sharing its CPU.
(There's other ways to figure out PID #s. This is just one way.)
top and htop are both ways to survey the server landscape. You can use this to see how busy the server is.
top -u mathaccountname
This will show you JUST your own processes. That's a nice way to make the output easier to understand.
htop simply shows CPUS in a more graphical format than top, which can be an overwhelming landscape of shifting numbers and columns.
free gives you a picture of the memory usage on a server. If you have ever had problems with a server, it might have been a memory error. Some people write poorly written scripts, and they might not function well, and eat memory as if they were potato chips. Or they might be writing their results to memory, intentionally, and the script is grabbing more and more as it continues.
This will tell you how much free memory is out there, not being used. While many of our servers have LOTS of memory, other people might be using it, so this could be a useful command to help you choose which server to use. If it seems like there's not much, choose another server.
Now, this expresses the amount of RAM as megabytes.
expresses the amount of available RAM in gigabytes. That might ultimately be more helpful.
time can be run against a specific command to figure out how much time, memory, and CPUs something might take.
time ls -R
shows how much time, memory and CPU this command (which is ultimately just a list command) will take.
Thus, let's say you are running multiple Matlab jobs; if you run this on just one, and notice it's taking multiple CPUs or a lot of RAM, you might reconsider what server would be best for this job.
You may need to type /usr/bin/time yourcommand
How to nice and renice your processes: https://www.nixtutor.com/linux/changing-priority-on-linux-processes/
12 tips for using top: https://www.tecmint.com/12-top-command-examples-in-linux/
How to use "time" to figure out how long a command line process might take: https://askubuntu.com/questions/53444/how-can-i-measure-the-execution-time-of-a-terminal-process/53446
All about Free: http://www.linfo.org/free.html