InternalName versus DisplayName

When creating columns (more commonly called fields) in SharePoint through the interface, you have to enter a name for it. This name is used throughout the lists and sites, included internally. Except when you try to change the name, it’ll only reflect on the outside. Internally the old name will be kept. This is because a field has two names: an internal name and a display name. When creating a field, you set both. When renaming it, you only change the display name. (There is actually no way to change the internal name afterwards)
But why is this a concern? Well, in the object model it can become quite vague when to use the internal name and when to use the display name. Here is a short list with some common methods and the name they need.

• SPFieldCollection[name] : SPField
  name: DisplayName
  unexistent: exception
• SPFieldCollection.GetField(name) : SPField
  name: internalName, displayName or internalName and displayName from the current context
  unexistent: exception
• SPFieldCollection.GetFieldByInternalName(name) : SPField
  name: internalName
  unexistent: exception
• SPFieldCollection.ContainsField(name) : bool
  name: displayName or internalName
  unexistent: boolean
• SPListItem[name] : object
  name: internalName, displayName or internalName and displayName from the current context
  unexistent: null
• SPListItem.GetFormattedValue(name) : string
  name: internalName, displayName or internalName and displayName from the current context
  unexistent: exception

If I find more relevant functions, I will update this list. On a related note, there also exists a static name. This is a name used by the field type. This is different from the internal name, as the internal name must be unique in its list and could have changed.