I’ve decided I need to become a programmer again. As an undergrad, and to a lesser extent as a grad student, I wrote code all the time. But since I started doing research, it’s been pencil-and-paper almost all the way through, with occasional dips into Mathematica or plotting programs.
That must end, so I’ve decided to learn Python. I just need something simple for number-crunching and graphics, and everyone in the know seems to have nice things to say about the language. (Secretly I would like to play around with genetic algorithms and cellular automata, but I’m not going to admit that.) I tried to get Fortran, my previous language of choice, up and running on my Mac … it didn’t go well.
So… any tips? Pointers to well-written resources and tutorials (online or in print) would be especially helpful. Keep in mind that the target audience is an aging theoretical physicist who hasn’t programmed in 20 years, and for that matter has been pretty much command-prompt free (working on my Mac) for the last five.
The things I admit in public on this blog, sheesh.
Hi Sean, we are developing (and soon releasing) a product targeted at exactly this sort of use case. It is based around Python but focused on people who aren’t so much looking to write software as much as get specific tasks done (like data analysis). If you are interested shoot me an email message and I’d love to get your thought and hear more about what you want to do.
Why no love for Sage? This makes me very sad. In case you missed my earlier comment, SAGE, SAGE, SAGE, SAGE, SAGE!
Get the box set, although the Life of Brian is my favourite.
My situation is very similar to yours. I was fluent in C (and Fortran, and Pascal, and …) when younger but haven’t programmed in anger for well over a decade. And by coincidence I started teaching myself Python about two weeks ago.
So far I’ve found it a very welcoming language. There is a natural and succinct syntax for the operations and data structures you need most often. In some ways it feels like having the best parts of C and Lisp in one place, and it definitely feels like it was designed by people who have actually absorbed the language design lessons of the past (unlike certain other languages I could mention…).
Despite the warnings above I’ve jumped straight into Python 3, since I’m not doing anything mission- or business-critical. My text of choice is “Programming in Python 3” (Summerfield).
Have you considered LabVIEW? It’s graphical, effective, and designed for scientists.
Consider VPython. It is Python plus a 3D graphics module called “Visual.” It has been used to create an interactive, calculus based, physics curriculum at North Carolina State University (http://www.matterandinteractions.org) in addition to projects on 3D systems and data visualization.
http://www.vpython.org
http://en.wikipedia.org/wiki/VPython
The Wikipedia entry states, “Real-time, navigable 3D animations are generated as a side effect of computations. This makes it easy to create simple visualizations, allowing programmers to focus more on the computational aspect of their programs.”
Thanks for all the input! Very helpful, especially about the most useful libraries for science. (I went and installed the enthought packages, which include scipy, numpy, and matplotlib, among others. Now to figure out what those do…)
I’m going to take a crack at Python itself, just because I would really like to have a simple all-purpose way of writing small programs when the occasion arises, and the distribution is already installed on my Mac. After an hour of fiddling, it’s amazingly easy to use, for which I am grateful. But I might check out SAGE at some later point.
I promise that any satellites I cause to crash will create minimal damage.
If you’re looking for a good resource of Python packages (not sure how much of astrophysics is relevant to your work) you may want to look at the resources page on astropython.org: http://www.astropython.org/resources
Also check out astrobetter.com for Python tips. There’s a lot of useful blog posts on Python there.
Everyone above has made excellent suggestions. The most important packages for Python are Numpy, Scipy and Matplotlib. If you need to visualize 3D data from simulations (common for cosmology) I’d recommend yt: http://yt-project.org/
Last but not least, using the Enthought Python Distribution (EPD) is highly recommended for new users. It’s free for those from academic institutes and it uses some features that will speed up Scipy significantly (Intel specific libraries for fortran wrapped files): http://www.enthought.com/products/epd.php
Sean, you can program in mathematica – loops and all. I’ve tried python and mathematica is, by far, still better if speed of execution is not critical but time invested in coding is.
Like someone else above said, python is not a serious language for scientific computing, but more of a scripting language when dealing with data structures and sets. Python is used in astronomy, along with IDL — you could also try IDL – but I have a feeling you’re not doing astronomy.
For normal stuff, stick with mathematica. If you need speed, use C/C++ or Fortran.
Once you know the basics, every few weeks, reread this:
http://python.net/~goodger/projects/pycon/2007/idiomatic/handout.html
When programmers know one language and switch to another, one of the most common problems is to keep to many old habits. Unfortunately, what might be a good habit in language A is often possible in language B, but a bad idea, or at least sub-optimal. It’s very easy to “write FORTRAN in python,” and I see a lot of it from astronomers, but the results are often painful. The hardest part of learning a language isn’t learning the syntax, it’s learning the patterns of thought that lead to using the language well.
Tom Aldcroft at the Harvard Center for Astrophysics wrote a pretty good walkthrough on how to use python and many subsidiary packages, plus it has some good examples as well. Many astronomers I’ve spoken to say python is the future of astronomy programming, and Tom has been writing a lot of packages to make this so.
http://python4astronomers.github.com/
To the few stating that “python is not a serious language for scientific computing, but more of a scripting language”, please recognize that it is 2011, not 1999, and update your opinion of python accordingly. Even as someone who actually does like Mathematica quite well, there are only a few specific use-cases I would recommend it over python.
It just isn’t, especially for high performance applications. If the extent of your scientific computing is a small MC run, or doing a bit of statistical or data analysis on your laptop, then Python is fine – just as Mathematica is — or IDL. My contention is Mathematica is faster (in time spent coding), and better for such ‘routine’ tasks, unless you’re an astronomer (IDL is more commonly used).
However, if you want to run a high performance DFT on a Tesla workstation, run some sort of Fisher matched filter analysis for GW signal hunting, or do a complex 3D magnetohydrodynamic simulation then you’re not going to use Python on your laptop. Compilers for C are much better tuned for specific architectures too, regardless of the level of language abstraction. I haven’t kept up with compilers, but a few years ago, the intel C compiler was much faster than third party compilers for C, or any other language, for x86 intel processors (maybe GCC finally caught up?).
Sean,
I’d also get a good IDE to work in, unless you already are an expert with vim or emacs. It will make your programming a lot more pleasant, if you’re used to tools from the old, old days — syntax highlighting, object properties, accurate code completion, context-specific documentation, debugging, etc.. ActiveState has a free version of their Komodo Python IDE which I would recommend. Alternatively, there are also a number of open-source IDEs for Linux which I suspect also have Mac versions; I’ve used Spyder which was solid, and Eclipse+PyDev which was comprehensive but a bit slow and clunky for everyday use. I’ve heard that Wing IDE and PyCharm are great commercial IDEs but I’ve never used them. Avoid the mini-editors that come with Python distribs (like IDLE.)
I also *love* ipython as a standalone shell for Python. Since it’s an interpreted language, you will probably often find yourself trying things out in a running Python shell before you write them into an actual program. iPython is far superior to the python shell for this… it has syntax highlighting, it lets you go back through old commands easily, it lets you easily inspect live objects and quickly get help and see the source of functions/objects, and it lets you “magically” use shell commands inside of Python.
—
numpy lets you do array computation FORTRAN-style in Python. That’s essentially what it’s for. Instead of having to write loops to go through element-by-element, with a numpy array you just write the array expression and it is automatically done element-wise. It also lets you use more powerful & general indexing schemes, it will “auto-promote” arrays to a higher dimension when needed, etc.
scipy gives you a bunch of useful mathematical libraries.
Python is incredibly straightforward, you should have no trouble with it. The official documentation at http://python.org/doc/ is excellent, in particular the language and library references are incredibly useful.
If you’re interested in python for math, you can use Sage, a python-based computer algebra system, which by now is better than Mathematica for some things (including programming, since it’s based on python). Sage is at http://sagemath.org . SAGE isn’t as straightforward as python itself, and if you want to install on a mac, you have to download the exact version corresponding to your version of MacOS.
Z, again you are showing a decades-old mindset. The first article I saw about using python to direct parallel simulations on supercomputers at LANL was in the late 90s. Nowadays the new versions of ipython have made casting parallel numercal tasks onto clusters almost effortless. You also apparently aren’t aware of pyrex, or cython, for easily generating efficient C extensions for those numerical tasks automatically, on the fly. And I guess you aren’t aware that the large numerical extension packages are all written in C and Fortran to begin with, and simply wrapped in python. Or that lots of people have been doing big-data on big-iron and putting their results on display at SciPy and other conferences for years. Having used both Mathematica and python I don’t really buy the programmer-efficiency argument, either. They both have excellent documentation, and offer clear learning curve advantages over C or Fortran or C++. But if you have practical concerns like interfacing with remote datasets or experimental hardware, or providing engineering guis for feckless graduate students, python is a clear win. Maybe if you are doing purely symbolic work I would recommend Mathematica, but Sage neatly wraps up Maxima, so I’m not even sure I could claim that in every case.
I mean, I like Mathematica. It’s a highly developed professional and polished product. If you or anyone else is productive with it, great. But the notion that python is a toy or wholly unsuited to scientific computing ignores the reality of scientists who are, in fact, using it exactly and effectively towards that purpose.
Oh for goodness’ sake – use a real language with a real IDE and a proper computational model behind it. The only language even approaching good enough to be worth criticising is Smalltalk; several free versions exist including perhaps the most popular one, Squeak. It has the virtue that it really is write once run everywhere.
http://www.squeak.org
For learning Python I’d give a vote to “Learning Python” from O’Reilly. You can use that while learning the language as well as a reference afterwards. Available in the traditional dead tree format as well as the more portable digital file. One has to admit that it does not quite reach the compactness of “Kernighan and Richie”, but Python comes with more stuff included… =)
In case you’re still here…
Evolutionary Computation (a unified approach) by Kenneth A. de De Jong
The table of contents and the first few pages are available at amazon.
Sean,
You really, really need to take a look at http://www.jsoftware.com for a free and fantastic alternative to Python.
I’d heard all the good things about Python and resolved to educate myself in it. I didn’t get very far, but far enough to encounter one slightly disturbing aspect, which is that the people developing Python don’t seem to believe in backwards compatibility. In other words you can have a piece of code that works fine in Python 2.n, but fails to run in 2.m where m > n. This seems poor. Fair enough to add new bells and whistles, so that a recent piece of code won’t run on an old machine, but old code should always work. I can dig out fortran that I wrote 20 years ago and it still does something (on a MacBook, moreover: just use g95). So this left me nervous about a wholesale shift to Python as my main way of doing things, since I could imagine spending months building up a repertoire of code to do all the things I commonly need, only to find that Python stick out a new version and suddenly none of it works any more.
I thought the first two comments about python not being a powerful tool for scientific programming were trolls, but after continuing to read the thread I think they are sincere.
I am following up to reiterate what Bryan says above. We’ve had people from Argonne show up to our usergroup to discuss python running on bluegene. We’ve had the primary dev of matplotlib come give a talk on it, and discuss why they created it and how they’ve used it, for example http://python.mirocommunity.org/video/1136/matplotlib-the-popular-2d-plot
Heck, I couldn’t remember some of the talks I’ve gone to, so I googled and found that the scipy website breaks out examples in to categories for you. http://www.scipy.org/Topical_Software
I strongly suggest looking at Langtangen’s book on computational scripting with python. I do scientific computing in python and have an earlier edition of this book which was, out of the many good books and resources for python, the best by my measure. I believe he has a lower-level primer out as well.
I have very much enjoyed the approach of this book: http://learnpythonthehardway.org/
One program to use: Pylab
It mixes Python with Numpy (arrays & related math), Scipy (numerical tools), and matplotlib (plotting) all into one easy to use package.