A DOM element is an object, a thing in memory. Like most objects in OOP, it has properties. It also, separately, has a map of the attributes defined on the element (usually coming from the markup that the browser read to create the element). Some of the element's properties get their initial values from attributes with the same or similar names (value gets its initial value from the "value" attribute; href gets its initial value from the "href" attribute, but it's not exactly the same value; className from the "class" attribute). Other properties get their initial values in other ways: For instance, the parentNode property gets its value based on what its parent element is; an element always has a style property, whether it has a "style" attribute or not.
Let's consider this anchor in a page at http://example.com/testing.html:
Some gratuitous ASCII art (and leaving out a lot of stuff):
| a ¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†|
| href: ¬†¬†¬†¬†¬†¬†"http://example.com/foo.html"
| name: ¬†¬†¬†¬†¬†¬†"fooAnchor" ¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†|
| id: ¬†¬†¬†¬†¬†¬†¬†¬†"fooAnchor" ¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†|
| className: ¬†"test one" ¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†|
| attributes: ¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†|
| ¬†¬†¬†href: ¬†"foo.html" ¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†|
| ¬†¬†¬†name: ¬†"fooAnchor" ¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†|
| ¬†¬†¬†id: ¬†¬†¬†"fooAnchor" ¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†|
| ¬†¬†¬†class: "test one" ¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†|
Note that the properties and attributes are distinct.
Now, although they are distinct, because all of this evolved rather than being designed from the ground up, a number of properties write back to the attribute they derived from if you set them. But not all do, and as you can see from href above, the mapping is not always a straight "pass the value on", sometimes there's interpretation involved.
When I talk about properties being properties of an object, I'm not speaking in the abstract. Here's some non-jQuery code:
var link = document.getElementById('fooAnchor');
alert(link.href); ¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†// alerts "http://example.com/foo.html"
alert(link.getAttribute("href")); // alerts "foo.html"
(Those values are as per most browsers; there's some variation.)
The link object is a real thing, and you can see there's a real distinction between accessing a property on it, and accessing an attribute.
As Tim said, the vast majority of the time, we want to be working with properties. Partially that's because their values (even their names) tend to be more consistent across browsers. We mostly only want to work with attributes when there is no property related to it (custom attributes), or when we know that for that particular attribute, the attribute and the property are not 1:1 (as with href and "href" above).
The standard properties are laid out in the various DOM specs:
These specs have excellent indexes and I recommend keeping links to them handy; I use them all the time.
Custom attributes would include, for instance, any data-xyz attributes you might put on elements to provide meta-data to your code (now that that's valid as of HTML5, as long as you stick to the data- prefix). (Recent versions of jQuery give you access to data-xyz elements via the data function, but that function does more than that and if you're just dealing with a data-xyz attribute, I'd actually use the attr function to interact with it.)
The attr function used to have some convoluted logic around getting what they thought you wanted, rather than literally getting the attribute. It conflated the concepts. Moving to prop and attr is meant to de-conflate them. There will be some brief confusion, but hopefully a better understanding of what's really going on going forward.
Some time kicking around the specs above, and experimenting, should help clear some of this up.
Update: jQuery 1.6.1 changes the attr function again, in ways that the jQuery team say mean most code that currently uses attr will continue to work, even if prop would technically be preferred. Details in the jQuery 1.6.1 blog post.