So called Android fragmentation is caused by the large number of Android device types and operating system versions deployed in the market. It's the flip side to the benefit of having a wide variety Android devices and capabilities.
There are currently over a thousand different types of Android devices on the market. It's simply not possible to support all of these without accomodating variations in device design.
Google provides a number of tools for addressing the compatibility necessary to serve this diverse universe of devices. These include:
Android 4.0 combines the support of tablets and handsets into a single release.
In Android 4.0 Google has made the inclusion of the unmodified Holo theme family a compatibility requirement for devices running Android 4.0 and forward. This will provide a level of uniformity across device implementations.
Provides mechanisms for supporting multiple screen sizes and densities.
Google has recently developed a set of classes to give the Action Bar backward compatibility for pre Android 3.0 devices. These classes invoke the action bar design pattern on pre-API 11 devices and the built-in Action Bar on devices supporting API 11 or greater.
This library is available through the SDK Updater. It gives backward compatibility for the new Fragments feature.
Google provides guidelines on achieving compatibility across devices.