If you haven't heard of Linaro, then now would be a good time to take notice. The Linaro project was recently able to tweak Android 4.0 code in a way that improved its performance on a Galaxy Nexus from 30% to a whopping 100% in some cases!

Linaro is a non-profit organization that was founded in 2010 by a consortium of hardware and software companies, including ARM, Samsung, TI, and Canonical in order to research maximizing the efficiency of Linux on ARM processors and more. Here's a quote with more details about the project/group,

The group is closely aligned with Ubuntu, but the improvements that it is driving offer benefits for the broader ecosystem of platforms and distributions that are deployed on ARM hardware. They have done a lot of work upstream in GCC (the GNU Compiler Collection) to open the door for better ARM optimization in Linux and other open source software.

Linaro’s GCC improvements have been producing measurable performance advantages over Google’s stock Android environment and build toolchain since late last year. Google is reportedly accepting some of these improvements in the upstream Android Open Source Project and independent developers are also looking to put them to use.
Apparently, these performance tweaks don't just show up in benchmarks either. Supposedly, subjective performance evaluations show a dramatic difference in app loading, faster scrolling, and a much smoother UI. The improvements are so impressive, they are already being adopted by the developer community. Here's another quote with some final details,

The code changes are being merged into CyanogenMOD 9 right now, which means that future nightly builds will take advantage of the optimizations. The Android Open Source Project (AOSP) is also accepting some Linaro submissions into its upstream, so there’s potential that Jellybean, or perhaps later versions of Android, will see these benefits.
It's amazing what a few improved algorithms and code tweaks can do to enhance the performance of an OS on its hardware. It will be interesting to see how quickly these engineers get snapped up by Google or other companies to improve things on a larger scale...

Thanks for the tip, metalspring!

Source: MobileSyrup and ArsTechnica