Saturday 26 February 2011

Maverick Installer, Linux kernel (2.6.31.14.20, February 2011)

The updated Maverick installer and "rescue image" is now online on PowerDeveloper.

Firmware Update

The firmware update has been moved to a seperate card for ease of use and to work around VFAT bugs in U-Boot. See the previous blog post for details on the changes.

Kernel Update (2.6.31.14.20 - master branch)

  • Operates on ALL Efika MX models (TO2, TO3, Smartbook)
  • See previous blog posts for details..

    Maverick System Update

    • Operates on ALL Efika MX models (TO2, TO3, Smartbook)
    • Installer:
      • Asks before installing and can drop back to a terminal for "rescue card" operation
      • Creates a dedicated swap partition instead of a swap file. It's configured to be 576MB at the end of the SSD.
    • OEM Configuration:
      • Encrypted home directories (and encrypted swap) now work
      • No longer requires a password for the "oem" user keyring connecting to wireless
    • Generally better performance due to kernel updates
    • Accelerated 2D and 3D drivers installed
    • Rollback to Ubuntu mainline versions of some packages

    All updates released by Ubuntu until today have been applied so there should be very little in terms of packages that are out of date. All systems (Smarttop TO2, TO3 and Smartbook) will boot into the installer, copy in the image to the PATA disk, reboot and ask for Language, Timezone, Keyboard settings, Username, Password, Machine Name and suchlike. This will replace the old "oem" user with one of your choice.

    You may need to add yourself to the "Audio" and "Video" groups to enable sound and webcam access. You can do this as you log in to the system for the first time through the Users and Groups control panel item.


    Okay, I got the maverick-installer.img.xz file, how do I install it to my system?

    Simply extract the archive (WinZip, 7-Zip, or command line at your option) and use your favorite utility here to copy it to a 2GB SD card. The utilities recommended are "dd" (or dcfldd), "flashnul" on Windows, or on an existing Ubuntu installation the Ubuntu USB Image Writer (you can find a list of imaging utilities here).

    As an example with "dd", you can do the following:

    xz -dc maverick-installer.img.xz | dd of=/dev/mmcblk0

    Once you have this image on an SD card simply power off your system and insert the SD card. Power it back on and the system will install automatically and when complete, shut down automatically. You may then remove the SD card and boot into the installed system. It really is that simple.


    I don't want to trash my system, can I just get the packages?

    Yes, there is a package repository going up right now, wait for a blog post later today.

    Monday 21 February 2011

    Efika MX Kernel Roadmap

    As an aside, the kernel mentioned in the last post will probably be our last release on the .31 series. The Linux kernel maintainers no longer really care about it, and even Freescale have moved on. We considered it important, however, to actually fix bugs where they stand rather than dragging users through new kernels unecessarily.

    Here is our current kernel roadmap after the release of 2.6.31.14.20 (current schedule: Friday 25th February 2011).

    2.6.35.11 (longterm support)

    We need to support the 2.6.35 series while Freescale uses it as the basis for their BSP releases. This is mainly because while a lot of support for the platform is going into mainline as I type this, it is not comprehensive enough to adequately support customers or the multimedia features we require (case in point, the Flash player).

    Since 2.6.35.11 and forwards will be long term supported by the Linux Foundation and maintained by distributions, and because it is the basis of the Ubuntu 10.10 (Maverick) distribution in particular, it makes sense to move all our improvements and drivers to 2.6.35.11 and forward, so we can maintain support for the BSP components required.

    2.6.35.x will be maintained for customers and internal testing, and for the MX51 platform, but we will probably only make very minimal number of releases before it is "dropped" for mainline. This will mostly depend on where Freescale go with their kernel support (a potential 2.6.38/2.6.39 BSP pending at some unknown date). Whether we engage in this kernel version for MX53 is still a decision to be made. We will, however, keep supporting TO2 boards.

    Development on 2.6.35 will start next weekend (Saturday 26th February 2011), or sooner :)

    Mainline (2.6.38 or 2.6.39)

    Using the sterling work of Sascha Hauer and other developers (hi Arnaud, Amit!) there is now support for the vast majority of the Efika MX platform requirements - Smarttop and Smartbook can be built using Sascha's imx development tree and some patches from Arnaud Patard. However it does not support some important features mentioned above, which makes shipping it a difficult decision. We do want to support this kernel, and we do want to support Linaro in getting our platform into the standard Linaro tree, since Ubuntu and other distributions will soon start using it as a basis for ARM platform support.

    We will work with mainline (Sascha etc.) and real mainline (Linus, GregKH etc.) to bring in the improvements we need to get the best platform support for ARM Smartbooks and other devices as we possibly can. This will happen immediately after we release our first .35 kernel, and how well this work goes will determine how quickly we drop support for .35.

    We are already picking around these trees and have booting kernels on Gentoo (good work, Steve). Any releases we make will be for DEVELOPMENT AND TESTING PURPOSES ONLY. We will not ship it on any installer or product by default, but just so we can keep everyone inline with the development track as we move forward.

    Major feature additions required for shipping (not conclusive):

    • i.MX51/53 VPU support in kernel (and userspace libva support for the decoder/encoders). This will also include getting JPEG decoding done in the VPU, since this is actually possible up to a certain size limitation.
    • i.MX51/53 IPU support under DRM (GEM allocator, Connector/Encoder/CRTC)
    • Vastly improved EDID support and parsing in the mainline kernel (even under DRM, the parsing is terrible, if they are to move all this into the kernel for KMS and take control away from Xorg, they have to do a better job. Saleem Abdulrasool is spearheading this for us, though, and he's done an excellent job so far..)
    • i.MX51/53 GPU support - possibly a non-mainlined patch to add GSL with a DRM wrapper. Lots of licensing issues to deal with here.
    • i.MX51/53 Security Engine support. This needs to be a kernel crypto/hashing driver and it needs to be able to be accessed via /dev/crypto so that OpenSSL and so on can adequately use it.
    A lot of this will also require on userspace support for several very important features for the above (video decoding especially, PulseAudio and libc optimization for NEON, that kind of thing).

    Future

    There's always room for improvement.. we have some proposals for research projects that developers may be interested in looking into, that nobody seems to have picked up yet:

    • Can we adequately use NEON in the kernel* to
      • speed up memory copies from userspace?
      • speed up TCP/IP checksumming?
      • speed up hashing?
      • speed up RAID (XOR)?
      • (requirements: ARM doesn't seem to have the use-FPU-in-kernel functions that PPC and x86 have to arbitrate access to this)
    • Are there points in the kernel where ARMv7 architecture additions are not being used, or an algorithm or function is implemented as a core library function of the kernel (in lib/ or kernel/ in the root of the tree) where other platforms implement optimized versions?
      • Silly things to look at might be things like popcnt.
      • Do these functions get used enough that they would produce a noticable speed improvement to make the work worthwhile?

    * footnote: Freescale did write patches for 2.4.x kernels back in the day, for AltiVec support at least a couple of these things (see libmotovec). Other developers pushed AltiVec and SSE support in the kernel for RAID management, so kernel support of these units does happen on some platforms. NEON is helped in this matter by having the same exception-less design as AltiVec and SSE (and AVX) which means you won't be causing kernel panics for "unhandled FPU exceptions" (as would be the case with a traditional FPU. As a side note, VFPv3 can run exception-less too, so that is also an opportunity..)

    Sunday 20 February 2011

    State of the Kernel (git, master branch)

    Just a blog to basically summarize the work that has been done since the last PowerDeveloper release of the kernel and the "January" Maverick image which ships on current systems. We are prepping an update to be released next week with a real package repository and 3D drivers for all. Stay tuned. In the meantime, here's the kernel work log.

    (to be 2.6.31.14.20 - master branch)

    Core
    • Operates on ALL Efika MX models (TO2, TO3, Smartbook) with a single kernel
    • The holy grail: merging of the two platforms is complete! One kernel will boot both systems. Much less is done on initialization for both platforms and boot is faster. Several redundant portions of code have been removed.
    • Serial driver updates (console polling and KGDB support)
    • A brand new battery driver
      • All properties are exported - GNOME battery display now works as expected
      • Notification of loss of conditioning and battery initialization data (batteries that report this need to be gently nudged to re-learn charge rates etc. for reliable operation)
    • Input subsystem rewritten to use real input handlers and events rather than gpio-keys which turns out not to be ideal. This improves:
      • Power key on Efika and Smartbook now provokes GNOME into asking if you want to shut down
      • Suspend and Resume are more reliable
      • Power, Wireless Switch and Lid events are watchable using "input-events" tool, as are battery insertion, low alarm and AC adapter detection (battery driver is hooked into this support)
      • Caps lock light on Smartbook is now a configurable LED trigger (if you don't care about caps lock, try using it for "mmc0" or "mmc1" status)

    Display
    • A new and greatly improved HDMI driver on Smarttop.
      • More CEA modes supported
      • Reinstated culling of incompatible modes
      • For better compatibility with TVs and real HDMI monitors, the resolution selection is as follows:
        • Look for a mode matching a low field rate 1080p mode (24Hz or 30Hz progressive)
        • If not found, look for a mode matching 720p (any field rate)
        • If not found, selects resolution close to the 1st detailed timing in the EDID (native panel resolution), but this mode may have been culled because it is out of range
        • If there is no suitable mode, force 1280x720@60
        • Override "teneighty" and "seventwenty" modes with those switches to the siihdmi driver.
      • HDMI vs. DVI detection is more reliable
      • Export of EDID property in sysfs (/sys/class/graphics/fbN/edid)
      • Hotplug support and D3 power down
      • Configures SPDIF audio on Smarttop. You may need to reconfigure PulseAudio to get audio out on your TV or AV receiver.
    • Display code has been cleaned up, Smarttop platform code no longer interacts with the framebuffer clocking or display mode selection. IPU is no longer set up before a real screen mode is available.
    • 32-bit framebuffer on Smarttop (override with bpp=)
    • Backlight support on Smartbook has been improved to use fewer steps for the backlight brightness and when the backlight is at 0%, the backlight is powered down to support PixelQi screens better and save a little battery life on screen blanking.
    • Backlight keys on Smartbook now work

    Updates from Mainline/Other kernel trees
    • ecryptfs and unionfs and securityfs and security key retention (for encrypted home dirs)
    • VPU and IPU updates from Freescale BSP 11.01.00 (stability)
    • Re-enabling DVFS and DVFS-PER power management
    • Audio driver now turns clocks off when it's not playing audio and has dual FIFOs for input and output. Kernel scheduling has been updated to enable rtkit to properly configure realtime process support for pulseaudio solving most audio playback issues
    • Redundant platform-specific code removed from core Linux parts (input, etc.)
    • Smarttop ethernet should now be a bit more reliable with VLAN tagging and no longer perform unaligned accesses
    • Scheduler updates to stop PulseAudio from complaining about SCHED_RESET_ON_FORK
    • Wireless driver updates
    • ARM core updates
    Many many thanks go to Steve Klimaszewski and Saleem Adbulrasool for their hard work on kernel components (especially HDMI and Battery). Anyone willing may compile the kernel from the master branch on the proviso that they may not get a working system, we will be precompiling an optimized and fully packaged kernel very soon, along with all the userspace driver updates required for best performance.

    Thanks for your patience!

    Thursday 17 February 2011

    One Kernel to Rule Them All

    This is a broadcast message to ALL Efika MX Smarttop and Smartbook developers and users who like to tinker with the kernel.

    We finally completed our work on merging the two platforms together. It is now possible to build ONE kernel which supports both systems - one prefix (-efikamx) and no fiddling building two kernels slowly and natively to get your work done.

    All developers are encouraged to switch to the "master" branch of the repository and reset their config to match the mx51_efikamx_defconfig (you can do this quite quickly and easily by backing up your current configuration and then using "make mx51_efikamx_defconfig" in the root of your kernel tree).