Dec 22, 2008

Compiling Qt apps in Windows with MinGW

I love Qt. I haven't actually used it much, as I haven't written many programs, but I've seen enough stuff written about it in developer blogs to have a fanboyish appreciation for the cross-platform toolkit. But aside from installing KDE for Windows back in it's beta days, I hadn't had the opportunity to use the "cross platform" bit until yesterday. I needed to compile screenie for a Windows environment. I knew what I was getting into, having compiled C and C# programs in the past. (I'm a big fan of interpreted languages; compiling always seems to waste hours of development time and involve enormously frustrating obscure compiler and linker errors.) But I loved learning about the process, and particularly discovering that I was capable of overcoming all my compiler errors with my current knowledge and Google. I got to understand Qt (and Windows executables, for that matter) on a more intimate level, solidify in my mind what .dll's are, and what dynamic and static linking are. After multiple attempts, I ended up with a standalone executable that ran perfectly on XP and Vista. It's an amazing feeling, knowing you started with hundreds, maybe thousands of text files - barely contained chaos! - and ended with a single file - pure simplicity. Plus, I may be the first person to compile screenie for Win32, making my work useful to others. Hopefully the author will make it available on screenie's website.

And now for the boring stuff if you're reading this for the human interest aspect; or the interesting part if you're reading this for the "How To" aspect. After reading several articles about cross-compiling Qt, I decided not to attempt that, and just did the whole process in Windows XP.

How to compile a C++ Qt4 application for Windows
Obviously, these directions are specific to compiling screenie. If you're using this as a guide for compiling something else, use your imagination as needed to adjust the steps.
Where applicable, I specify the generic download page for each application, as well as the specific binary version I used.
1. Download and install Qt4 with MinGW
(http://trolltech.com/downloads/opensource/appdev/windows-cpp)
I used version: ftp://ftp.trolltech.com/qt/source/qt-win-opensource-4.4.3-mingw.exe
Since the source for screenie is a Git repository, I figured we need git. Update 2008-12-26: There is a Download button on the screenie Github page to download the source as a zip file. Extract it to C:\screenie and jump to step 5.
2. Download and install msysgit
(http://code.google.com/p/msysgit/downloads/list)
I used version: http://msysgit.googlecode.com/files/Git-1.6.0.2-preview20080923.exe
3. Run Git Bash and execute git clone git://github.com/ariya/screenie.git
4. Copy the downloaded source directory to c:\screenie
5. Open the Qt 4.4.3 Command Prompt from the start menu.
It's like a normal command prompt, except with some environment variables and bash scripts set. (For instance, it appears to alias make to mingw32-make, which saves typing.) It is assumed you are using the Qt Command Prompt for all the steps below involving a prompt.

By default Qt installs itself for compiling with dynamically linked libraries. This means you have to copy the compiled executable, as well as a bunch of .dll's to each computer you install it on. It's not really a problem: just put them all in a zip file, unzip it on the Desktop, and it runs nicely. But if you're like me, and want a single executable with no dependencies, scroll down to the statically linked bit.

Dynamically linked (.dll dependencies, distribute as .zip file)
6. cd C:\screenie
7. Issue qmake -spec "C:\Qt\4.4.3\mkspecs\win32-g++"
For some reason, qmake seemed confused out-of-the-box, and I had to fix several things in the makefile. I used the Find and Replace feature in Crimson Editor (my favorite Windows text editor.)
8. In Makefile.Release replace:
"iwmake\build_mingw_opensource" with "Qt\4.4.3"
(There were 8 instances)
"QtGui" with "QtGui4"
"QtCore" with "QtCore4"
(Both in the LIBS line)
The same substitutions would be needed to fix Makefile.Debug, except instead of QtGui4 and QtCore4 it's the debug versions QtGui4d and QtCore4d. But you don't need to build the debug version anyway.
9. Issue make.
The compiled executable appears in C:\screenie\release. Test the program and see if it works. For me, Screenie worked on Windows! Yea!

Following the advice on the Trolltech website, I used Dependency Walker to find what .dll's the executable requires. I copied those .dll's to the release directory. In screenie's case, Dependency Walker listed QtCore4.dll, QtGui4.dll, msvcrt.dll, and kernel.dll. Kernel.dll is already installed on every windows system so no need to distribute it. Msvcrt.dll is the Microsoft C Runtime Library - probably installed on people's computers, but I included a copy anyway, along with QtCore4.dll and QtGui4.dll. I had to include one more .dll that Dependency Walker did not find, a copy of c:\MinGW\mingwm10.dll. (See note in static compiling.) Also, any plugins used need to be copied to the release folder. In this case, screenie needs the image plugins to read JPEG or GIF files, so copy the contents of c:\Qt\4.4.3\plugins\imageformats to c:\screenie\release\imageformats. You can delete the .dll's whose names end in the letter "d"; they are debug versions and take up a lot of space, and are not used if you compile for release. Copy the release folder to the Desktop, give it a better name (like "screenie"), zip it, and voila! Copy it to another computer and hopefully it will run.

Statically linked (no dependencies, single file to distribute)

To my dismay, compiling statically was not simple. To begin, we must recompile Qt itself.
6. Using the Qt 4.4.3 Command Prompt from the start menu...
cd C:\Qt\4.4.3
configure -static
This takes a long time. I read later to do a distclean before rebuilding... but I didn't and it built OK. However, I wonder if this is why later qmake decided to link to the old .dll's ending in 4.
7. make sub-src
This takes a really long time. (By this point it was past 3:00 am, so I left it running while I slept.)

When Qt is statically linked, it cannot dynamically load plugins. (Turns out it's an artificial restriction that's easily overwritten.) But we want those image plugins statically compiled into our final executable. To make that happen you have to do the following voodoo. Luckily it's straight from the Trolltech documentation.
8. Add Q_IMPORT_PLUGIN() statements for each plugin to the c++ code.
I added the lines:
Q_IMPORT_PLUGIN(qjpeg)
Q_IMPORT_PLUGIN(qgif)
to screenie.cpp

(There are plugins for some other image formats, like .tiff and .ico but I didn't think most people would need more than .jpeg, .png, and .gif. PNG support is built into Qt so it doesn't have a plugin.) These lines of code are necessary to compile it statically, but I don't know yet if those lines cause trouble if you try to compile the code dynamically.
9. Add a QTPLUGIN line to the .pro file
I added the lines
QTPLUGIN += qjpeg qgif
CONFIG += static
to screenie.pro
(The CONFIG one is a guess... I actually got it from a page on compiling custom plugins, but figured we want all things static, so why not throw it in? Not gonna make them less static.) Voodoo done.
10. cd C:\screenie
make clean
qmake -config release
This time, it seems to not have the iwmake\build_mingw_opensource problem, but it does need for the 4 from the library names to be deleted. When compiling statically, the library names used do not have the 4 in them. I'd love to know why Trolltech thought that was necessary. (Funny how the 4's weren't there when I needed them, and now are there when I don't want them!)
11. In Makefile.Release, in the LIBS line, replace -lqjpeg4 and -lqgif4 with -lqjpeg and -lqgif.
Delete mthreads because it's an unneeded dependency on mingw10.dll which isn't compiled statically. Apparently Qt doesn't use it, so I don't know why qmake adds it by default. If you don't delete -mthreads, you get an error saying mingw10.dll can't be found when you run it on other computers.
12. Delete all instances of -mthreads in Makefile.Release
13. Now back in C:\screenie issue make
cd release
You should now have a statically compiled executable c:\screenie\release\screenie.exe. See if it works! If everything went according to plan, you can run that file on any Windows XP / Vista computer and use Ariya Hidayat's awesome screenie!

But no! You're a perfectionist. You want more. The screenie.exe file is about 10 MB. Let's see if we can shrink that! I got these tips from http://www.qtforum.org/post/85635/static-compiling.html#post85635
14. From the release folder, issue strip screenie.exe
This is a command from Qt that removes unnecessary parts of Qt from statically compiled files. Sadly, this didn't reduce the file size much, if any. (Perhaps Qt does that behavior by default now.) But there's another thing you can do to reduce file size.
15. Download and install the Ultimate Packer for eXecutables (http://upx.sourceforge.net/)
I used version: http://upx.sourceforge.net/download/upx303w.zip
I had to manually unzip it to C:\Program Files and add it's directory to Window's Path environment variable.
16. Open cmd and cd to c:\screenie\release
Issue upx screenie.exe
Now it is only 3.58 MB. Sweet!

Sep 11, 2008

How to build a MythTV for under $300

If you search for a media center PC online or in a store, they tend to be quite expensive, often in excess of a thousand dollars. But you don't need that much money to get a working media center that lets you pause live TV, record shows, rip DVDs onto the hard drive, and more. I did it for under $300, and with a little bargain hunting, you can too!

Here's what I have and how much I paid (rounded to the dollar):
$39 - ECS 761GX-M754 motherboard (eBay)
$10 - AMD64 3000+ CPU on sale at Fry's because the 754 socket is being discontinued. Great time to buy one!*
$33 - 1GB Corsair RAM (Fry's)
$25 - PowerSpec TX366 mATX Case (Microcenter)
$25 - XION 450W power supply (Microcenter)
$33 - Hauppauge PVR-150 NTSC TV tuner (eBay)
$38 - ATI X700 (eBay)
$21 - 50' CAT5 cable (Home Depot)
$20 - 1 year subscription to Schedules Direct (needed if you want to get channel schedules in North America)
$27 - Windows MCE remote (eBay)
Some additional parts that I already owned:
$0 - Two 80GB Western Digital IDE hard drives
$0 - DVD reader
---------------
= $271

* I actually got several since they were such a steal... if you wanna buy one, leave a comment!

Now, you're probably thinking, "Hey, that's not fair - you didn't include the price of the hard drives or the DVD reader!" Frankly, if you're not the kind of person who has a some spare computer parts lying around your house, you should probably quit reading now, because despite many advances in usability, MythTV is still quite a challenge to install and configure. However, for the sake of thoroughness: you can get a 160GB SATA drive for $50 and a DVD-ROM reader from $20 at Microcenter. Plus, in hindsight, I believe a 300W power supply would suffice, so instead of paying $50 for the case and power supply, Microcenter sells the case with a 300W power supply for $34. And on Craigslist, which I discovered late in the process of buying things, people are selling old desktops for as low as $50, which is as much as I paid for the case and power supply alone, but would potentially get you RAM, hard drives, an extra keyboard, maybe even the DVD drive if you're lucky. I got a 20" CRT monitor for $15 on Craigslist, so I'm sure that if you live near a big city you could get a better deal on a bare bones PC than I did building from scratch.

Be warned, installing MythTV is not for the faint of heart. It took the better part of two weeks to get it configured the way I like it, and I am already pretty well versed in Linux and Ubuntu. Most of the first week was spent trying to make the ATI tuner card, the ATI remote, and a cheap Trendnet TEW-424UB USB wireless dongle work with Linux. (The USB wireless adapter worked fine with ndiswrapper on my laptop, but refused to connect on my desktop.) After a week I gave up, and researched the best, most highly recommended hardware for MythTV and got a Hauppauge card, a Windows MCE remote, and traded wireless for a CAT5 cable, which all work great. I had to mess around with the LIRC config files until I got all the buttons on my remote programmed the way I like, but that really wasn't too hard once I figured out how. Also, MythTV's internal player had extremely poor DVD playback on my setup, so I had to configure MythTV to use Xine instead (the MythTV wiki explains how to do that very clearly).

All of the parts I used are a few years old - you don't need "modern" hardware to run MythTV. The AMD64 3000+ was released in 2003 according to Wikipedia. The 80GB hard drives are both from an old computer that got replaced years ago. Using the Hauppauge card with its hardware MPEG-2 encoder, standard definition analog TV takes up 2.2GB per hour; one drive is formatted with XFS and has 75GB of free space, the other is ext3 and has nearly 70GB free. That's over 65 hours of TV! (Mythbuntu itself appears to only take about 2GB. The formatted capacity of ext3 is less than XFS apparently.) As for the other hardware, I don't necessarily recommend the power supply I got - I find the fan rather loud and annoying. And I'm not sure where I stand on ATI vs Nvidia when it comes to Linux. But I definitely recommend the Hauppauge if you're recording analog. You might want a PVR-500, which has two tuners in it, if you want to be able to watch live TV while recording another show, or record two show at once. I figure the chances of two good shows being on at the same time, with no reruns that could be recorded later, are so slim all I need is one for now. Since I'm stuck with college cable, I doubt I'll get any digital or HDTV for a while, so I stuck with an analog tuner.

So if you've got a lot of free time and $300, go ahead - build yourself a MythTV! Record shows and skip through the commercials! Never worry about when your favorite show is on; MythTV can find and records your show on any channel at any time. Try programming a VCR to do that! MythTV also remembers which episodes it's previously recorded so it doesn't fill up your hard drive with reruns you've already seen. You can rip your DVDs to the hard drives. You can have multiple frontends, connected to a single MythTV backend by ethernet. The MythWeb plugin even lets you stream your recordings over the Internet so you can watch them on any computer. And all sorts of cool things. Useful resources definitely are the MythTV wiki, the Mythbuntu forums, the MythTV wiki, and did I mention the MythTV wiki? Really, the MythTV wiki is the most useful thing I encountered. FYI, I tried MythDora and KnoppMyth - MythDora seemed really nice - but stuck with Mythbuntu simply because I'm use to Debian/Ubuntu systems.

Here's some more pictures. I printed the MythTV sticker in the above picture by using the unused portion of inkjet DVD label paper.

The MythTV home screen with the "neon-wide" theme.

The MythTV program guide.

For fun, I re-branded the Windows Media Center Remote.
(You can guess what logo was there originally.)
So that's it.

Aug 25, 2008

Large numbers are truly incredible

Here's a favorite example: every time you drink a glass of water, the odds are good that you will imbibe at least one molecule that passed through the bladder of Oliver Cromwell. It's just elementary probability theory.
--Richard Dawkins, The God Delusion, pg. 366

That's such a fascinating claim I had to do the math myself. Obviously, the water cycle is too complex to model exactly, but this is a guesstimate after all. To make things extra hard, let's bias all the numbers by rounding them to weaken the chances of this happning.

What number shall we use? How Stuff Works says there are 326 million trillion gallons of water on Earth. Let's round that up to 10^21. Let's assume an 8 oz glass of water for this experiment. And, according to the Wikipedia birth and death dates for Oliver Cromwell and a date subtractor, Cromwell lived 21,681 days. Again, let's make this more challenging, by assuming he only peed 1 fl oz a day. (One source I found said people pee 1.5 - 2 liters a day!) The rest are just conversion factors, which are not exact and of course depend on the temperature of the water. But the difference turns out to be many magnitudes, so the conversion factors only need to be correct to within one magnitude for the premise to hold.

1. How many mols H2O are in an 8 oz glass?

8 fl oz * 29 grams / fl oz * 1 mol H2O / 18 grams = 12.8 mol H2O, which we'll round down to 10 mol H2O

2. How many mols H2O are on the Earth?

10^21 gallons * 3.78 L / gal * 1000 grams / L * 1 mol H2O / 18 grams = 2.1 × 10^23 which we'll round up to 10^24 mol H2O

3. How many mols H2O did Oliver Cromwell pee?

21,681 days * 1 fl oz / day * 29 grams / fl oz * 1 mol H2O / 18 grams = 34,930.5 mols H2O which we'll round down to 10^4 mol H2O

So what does that give us? The percent of Cromwell's pee to the total water of the world is: 10^4 / 10^24 or 1 part per 10^20. That seems like a pretty small percentage. Yet in one glass of water, there will be

10 mol H2O * 6.0221415 × 10^23 molecules / mol = 6.02 * 10^24 molecules

So from a sample where we have a 1 in 10^20 chance of getting a molecule of Cromwell-water, we take 6 * 10^24 samples. Heck, by these probabilities, we'd have a likelihood of getting roughly 6000 molecules of water that once passed through Cromwell's bladder. I personally doubt that all the world's water mixes evenly, and most of it stays at the bottoms of the oceans, making the pool of drinkable molecules drastically smaller. And I believe the Cromwell probably peed more than 1 fl oz a day. And although I certainly concede the possibility that many of the water molecules that Cromwell passed may no longer exist because they were broken up by photosynthesis, I think it's still possible to say that "the odds are good".

Apr 7, 2008

Dreaming of the future...

(Prompted by this blog post on Nepomuk.)

I love envisioning the future; it makes me feel like Leonardo Da Vinci, sometimes. It's one of those areas where I know I can't be much worse than even experts. Plus, since I'm not actually predicting, just dreaming, it's OK if my predictions don't come true.

dream of KDE 4.5 with Nepomuk+Strigi power!!!
*There's a new Contact application that brings together all your email, IM, VoIP, and Facebook contacts.

*Amarok uses Last.fm to find all the similarities between your music collection and your Facebook friend's music. When playing songs, it shows a little note, saying "Your [friend / co-worker / brother] Joe likes this song too." With a right-click, you can suggest a track to any of your contacts.

*When I type someone's name in a search box, it gives me options to send an IM (if they are online), email, message their Facebook directly.... That might be in a "Contact" pane. In another pane, it might show pictures of Joe, emails/IM logs from Joe, Joe's Facebook/Twitter status, and files on my computer from Joe. Another pane provides suggestions of files I might want to send to Joe such as recent pictures of Joe I took, or something I've sent to many of Joe's friends.

*I can type "old english essay politics" in the search box, and it will find it even though the words "old", "english" and "essay" are not in the filename or the document itself. It found the word "politics" (and associated words like "democracy") in the file, detected the name of my English teacher, knew which years I was her student, and thus knew it was "old" even though I updated the file it last week.

*Bob sends me an email asking when the Block Party and ice-skating event is. Since the Block Party is scheduled on Facebook and the ice-skating event on my co-worker's Google Calendar, KMail asks "Do you want to tell Bob that the Block Party is this Tuesday and the ice-skating is Mar 23?" and will write a template email for me. In KDE's internal calendar, Bob is marked as "possibly attending" those events. If he later replies, "Great! See you there" KDE marks him as attending, and updates my grocery list accordingly. (Earlier I got an email saying I was responsible for bringing punch to the block party.)

*Haha, and now we're getting to where the fridge orders more food... I always end up there somehow. :-)

Those last few examples bordered on AI, but regardless, this is going to be a great leap. Using an ontology and marking the relationships between files will help computers finally understand what files are. I might have a scan of piano music, a photo I took on vacation, a drawing my brother made in the GIMP, and a screenshot of my desktop, and all the computer sees is four JPEG images. Once that limitation is breached, I think computers will become much more helpful to people.

Jan 12, 2008

KDE 4.0.0

The long awaited and much anticipated 4.0 release of KDE has arrived!  Due to the KWin release notes - the COMPOSITE_HOWTO, to be precise - I got KWin OpenGL effects working!  They're fairly smooth on my Nvidia card, although they don't seem quite as smooth as I remember Compiz being.  Hopefully it, or X, or the Nvidia drivers, will improve to make KWin 4 w/composite snappy.  (It's not really "snappy" at this point like KWin 3 is.)  I haven't managed to install the KDE Edu or KDE Multimedia modules yet, and I am stuck with the Plasmoids that aren't in extragear, but hopefully Ubuntu will get those re-packaged for 4.0 soon.  (The current packages for those things depend on RC2, I think.)

KDE 4.0 is not perfect.  I wish I could put the panel at the top of the screen, for instance.  But I could certainly live with it (although I probably won't). I may stick with KDE 3.8 for a few more months, until I start writing plasmoids, or some other killer feature drags me into the new millenium. By the time Amarok 2 is released, and Decibal is added, etc, I'll probably convert.  At the moment, GTK apps look really bad, probably because I don't have the appropriate qt-theming-for-gtk-apps thing installed.  I'm sure Kubuntu will figure that out for Hardy at the latest.  I may insist on making it work sooner by doing the appropriate Google or IRC dance.  We'll see. I'm proud to have helped the KDE project, albeit in very small ways.  (One or two bug reports and a miniscule excursion in bug triage, which I plan to continue, time allowing.)  Mostly, just because KDE is the darn best collection of applications for the free desktop.  But also, because of the enormous vision of the project, encompassing everything from Windows and Mac ports to integrating Plasma with Linux MCE.  It's a bright future.  The present is bright too, obviously.

(If your wondering, why on earth I'm polluting the world with yet another random-user-posts-his-KDE4-experience, it's mostly because it's past midnight, and I get confused and ramble/waste time after 12:00.)