Note
Reference document for this section:
Contents
Download this page as:
“A portable way of using operating system dependent functionality.”
Current directory:
In [1]: import os
In [2]: os.getcwd()
Out[2]: '/Users/naw/VM-Share/GREAT-ITN/GITN-Meetings/GITN-School-Alicante-Jun13/degroote-doc-v2'
List a directory:
In [3]: os.listdir(os.curdir)
Out[3]:
['.DS_Store',
'_build',
'_build-v2',
'_static',
'_templates',
'anthony',
'cat.fits',
'conf.py',
'coryn',
'data.txt',
'data_new.txt',
'first_steps',
'index.rst',
'index.rst~',
'input_output',
'installation',
'ipython_directive.py',
'ipython_directive.pyc',
'irwin',
'make.bat',
'Makefile',
'numpy',
'plotting',
'py2rst.py',
'rst2py.py',
'some_program.f']
Make a directory:
In [4]: os.mkdir('junkdir')
In [5]: 'junkdir' in os.listdir(os.curdir)
Out[5]: True
Rename the directory:
In [6]: os.rename('junkdir', 'foodir')
In [7]: 'junkdir' in os.listdir(os.curdir)
Out[7]: False
In [8]: 'foodir' in os.listdir(os.curdir)
Out[8]: True
In [9]: os.rmdir('foodir')
In [10]: 'foodir' in os.listdir(os.curdir)
Out[10]: False
Delete a file:
In [11]: fp = open('junk.txt', 'w') # first create an empty file
In [12]: fp.close()
In [13]: 'junk.txt' in os.listdir(os.curdir)
Out[13]: True
In [14]: os.remove('junk.txt')
In [15]: 'junk.txt' in os.listdir(os.curdir)
Out[15]: False
os.path provides common operations on pathnames:
In [16]: fp = open('junk.txt', 'w')
In [17]: fp.close()
In [18]: a = os.path.abspath('junk.txt')
In [19]: a
Out[19]: '/Users/naw/VM-Share/GREAT-ITN/GITN-Meetings/GITN-School-Alicante-Jun13/degroote-doc-v2/junk.txt'
In [20]: os.path.split(a)
Out[20]:
('/Users/naw/VM-Share/GREAT-ITN/GITN-Meetings/GITN-School-Alicante-Jun13/degroote-doc-v2',
'junk.txt')
In [21]: os.path.dirname(a)
Out[21]: '/Users/naw/VM-Share/GREAT-ITN/GITN-Meetings/GITN-School-Alicante-Jun13/degroote-doc-v2'
In [22]: os.path.basename(a)
Out[22]: 'junk.txt'
In [23]: os.path.splitext(os.path.basename(a))
Out[23]: ('junk', '.txt')
In [24]: os.path.exists('junk.txt')
Out[24]: True
In [25]: os.path.isfile('junk.txt')
Out[25]: True
In [26]: os.path.isdir('junk.txt')
Out[26]: False
In [27]: os.path.expanduser('~/local')
Out[27]: '/Users/naw/local'
In [28]: os.path.join(os.path.expanduser('~'), 'local', 'bin')
Out[28]: '/Users/naw/local/bin'
In [29]: os.remove('junk.txt')
Call a simple command, wait for it to finish, and get the return code:
In [30]: import subprocess
In [31]: subprocess.call('chmod +x rst2py.py', shell=True)
Out[31]: 0
Get the output:
In [32]: output = subprocess.check_output(['ls','-all','-h'])
In [33]: print(output)
total 360
drwxr-xr-x 28 naw staff 952B 20 Jun 17:03 .
drwxr-xr-x 13 naw staff 442B 20 Jun 16:58 ..
-rw-r--r--@ 1 naw staff 6.0K 20 Jun 16:57 .DS_Store
-rw-r--r--@ 1 naw staff 5.5K 6 May 07:38 Makefile
drwxr-xr-x@ 5 naw staff 170B 15 May 13:02 _build
drwxr-xr-x 3 naw staff 102B 20 Jun 17:03 _build-v2
drwxr-xr-x@ 2 naw staff 68B 6 May 07:38 _static
drwxr-xr-x@ 3 naw staff 102B 6 May 09:51 _templates
drwxr-xr-x@ 3 naw staff 102B 14 Jun 09:52 anthony
-rw-r--r--@ 1 naw staff 68K 16 Jun 11:32 cat.fits
-rw-r--r--@ 1 naw staff 7.8K 20 Jun 17:03 conf.py
drwxr-xr-x@ 53 naw staff 1.8K 20 Jun 10:26 coryn
-rw-r--r--@ 1 naw staff 540B 6 May 16:23 data.txt
-rw-r--r--@ 1 naw staff 41B 6 May 18:22 data_new.txt
drwxr-xr-x@ 39 naw staff 1.3K 20 Jun 10:22 first_steps
-rw-r--r-- 1 naw staff 2.2K 20 Jun 17:00 index.rst
-rw-r--r--@ 1 naw staff 2.0K 20 Jun 10:35 index.rst~
drwxr-xr-x@ 8 naw staff 272B 6 May 18:21 input_output
drwxr-xr-x@ 17 naw staff 578B 10 Jun 15:17 installation
-rw-r--r--@ 1 naw staff 26K 20 Jun 17:01 ipython_directive.py
-rw-r--r-- 1 naw staff 20K 20 Jun 17:03 ipython_directive.pyc
drwxr-xr-x@ 3 naw staff 102B 20 Jun 10:34 irwin
-rw-r--r--@ 1 naw staff 5.0K 6 May 07:38 make.bat
drwxr-xr-x@ 59 naw staff 2.0K 16 Jun 11:41 numpy
drwxr-xr-x@ 29 naw staff 986B 15 Jun 16:39 plotting
-rw-r--r--@ 1 naw staff 1.4K 15 May 12:02 py2rst.py
-rwxr-xr-x@ 1 naw staff 3.0K 16 Jun 10:39 rst2py.py
-rw-r--r--@ 1 naw staff 255B 16 Jun 11:32 some_program.f
Communicate with the process (try for example with some_program.f, and compile it with gfortran some_program.f). Though you can download and compile the program in a terminal, you can just as well stay within Python:
In [34]: import urllib
In [35]: url = urllib.FancyURLopener()
In [36]: link = 'http://www.ster.kuleuven.be/~pieterd/some_program.f'
In [37]: myfile,msg = url.retrieve(link, filename='some_program.f')
In [38]: url.close()
In [39]: output = subprocess.check_call('gfortran some_program.f',shell=True)
In [40]: print(output) # output flag of zero means successful
0
Now run that program and read the first line from the standard output.
In [41]: p1 = subprocess.Popen('./a.out',stdout=subprocess.PIPE)
In [42]: print(p1.stdout.readline())
0.000000 **2 = 0.000000
You can also stop, continue and kill the program after importing the standard module signal:
In [43]: import signal
In [44]: p1.send_signal(signal.SIGSTOP) # temporarily halt the executing of a program
In [45]: p1.send_signal(signal.SIGCONT) # continue the program
In [46]: p1.send_signal(signal.SIGKILL) # kill the program
And finally we remove the compiled program again:
In [47]: os.remove('a.out')
Click to Show/Hide: How to communicate with a program during execution
Suppose we want to run a program, and check its output while it’s running. For this, we need to read the program’s standard output while it is running, wait for the next line to appear, and end the loop when the output stream is closed. This can be done with:
def line_at_a_time(fileobj):
while True:
line = fileobj.readline()
if not line:
return
yield line
Now, we can run the program and check the output. Suppose “myprogram” prints ERROR to the screen when it encountered an error, and we want to kill the program whenever that occurs:
>>> p1 = subprocess.Popen('./my_program',stdout=subprocess.PIPE)
>>> for line in line_at_a_time(p1.stdout):
if "ERROR" in line:
p1.send_signal(signal.SIGKILL)
Similarly, you can use subprocess.PIPE to send data to stdin.
Exercise: run some_program.f and communicate with it
Run some_program.f, and stop the program for 10 seconds when the number 4 is reached. Kill the program when the number 11 is reached.
Click to show/hide solution
import subprocess
import signal
import time
def line_at_a_time(fileobj):
while True:
line = fileobj.readline()
if not line:
return
yield line
p1 = subprocess.Popen('./a.out',stdout=subprocess.PIPE)
for line in line_at_a_time(p1.stdout):
number = float(line.split()[0])
if number==4.:
print("Reached 4: stop 10 secs, then continue")
p1.send_signal(signal.SIGSTOP)
time.sleep(10)
p1.send_signal(signal.SIGCONT)
elif number==11:
print("Reached 11: kill!")
p1.send_signal(signal.SIGKILL)
else:
print("Got: {:s}".format(line.strip()))
The shutil provides useful file operations:
- shutil.rmtree: Recursively delete a directory tree.
- shutil.move: Recursively move a file or directory to another location.
- shutil.copy: Copy files or directories.
The glob module provides convenient file pattern matching.
Find all files ending in .txt:
In [48]: from glob import glob
In [49]: glob('first_steps/*.rst')
Out[49]:
['first_steps/assignment.rst',
'first_steps/control_flow.rst',
'first_steps/functions.rst',
'first_steps/intro.rst',
'first_steps/modules.rst',
'first_steps/oop.rst',
'first_steps/python3.rst',
'first_steps/running.rst',
'first_steps/standard_library.rst',
'first_steps/types.rst']
In [50]: glob('first_steps/[ap]*.rst')
Out[50]: ['first_steps/assignment.rst', 'first_steps/python3.rst']
Note that by default the results aren’t sorted. If you want the output list sorted, use:
In [51]: sorted(glob('[hn]*.txt'))
Out[51]: []
In [52]: import os
In [53]: list(os.environ.keys())[:2]
Out[53]: ['TERM_PROGRAM_VERSION', 'LOGNAME']
In [54]: os.environ['PYTHONPATH']
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
/Users/naw/VM-Share/GREAT-ITN/GITN-Meetings/GITN-School-Alicante-Jun13/degroote-doc-v2/<ipython-input-54-24418705d836> in <module>()
----> 1 os.environ['PYTHONPATH']
/Library/Frameworks/EPD64.framework/Versions/7.3/lib/python2.7/UserDict.pyc in __getitem__(self, key)
21 if hasattr(self.__class__, "__missing__"):
22 return self.__class__.__missing__(self, key)
---> 23 raise KeyError(key)
24 def __setitem__(self, key, item): self.data[key] = item
25 def __delitem__(self, key): del self.data[key]
KeyError: 'PYTHONPATH'
In [55]: os.getenv('PYTHONPATH')
System-specific information related to the Python interpreter.
In [56]: import sys
In [57]: sys.platform
Out[57]: 'darwin'
In [58]: sys.version
Out[58]: '2.7.3 |EPD 7.3-2 (64-bit)| (default, Apr 12 2012, 11:14:05) \n[GCC 4.0.1 (Apple Inc. build 5493)]'
In [59]: sys.prefix
Out[59]: '/Library/Frameworks/EPD64.framework/Versions/7.3'
In [60]: sys.argv
Out[60]:
['/Library/Frameworks/EPD64.framework/Versions/Current/bin/sphinx-build',
'.',
'_build-v2']
sys.path is a list of strings that specifies the search path for modules. Initialized from PYTHONPATH:
In [61]: sys.path[-2:]
Out[61]:
['/Library/Python/2.7/site-packages',
'/Library/Frameworks/EPD64.framework/Versions/7.3/lib/python2.7/site-packages/IPython/extensions']