Windows batch to download and make a portable JDK

I am a fan of portable apps and don't like to install stuff unless there is no other choices, as installation on Windows platform usually will introduce some unexpected "side-effect".

JDK is one of the tool that I need but don't want to install during development. I want to perform some automated tasks to set up a portable development environment (for android development). But downloading and extracting the JDK in a workable state are a bit tricky. Here is what I have performed to automate the process under Windows environment.

To automate the task using Windows batch script, you will need to acquire the following command tools first.

• wget.exe
• 7z.exe and 7z.dll

To download the JDK (e.g. jdk-7u55-windows-i586.exe) from Oracle using wget.exe and extract the JDK to.\jdk_1.7.0_55, copy the following contents to a text file and save it as "getJDK.bat" in the folder where wget.exe and 7z.exe resided.

@echo off
"%~dp0wget.exe" --no-check-certificate -O "%~dp0jdk-7u55-windows-i586.exe" "http://download.oracle.com/otn-pub/java/jdk/7u55-b13/jdk-7u55-windows-i586.exe" --no-cookies --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie"
"%~dp07z.exe" x -y "%~dp0jdk-7u55-windows-i586.exe" -o"%~dp0jdk_1.7.0_55"
"%~dp07z.exe" x -y "%~dp0jdk_1.7.0_55\tools.zip"  -o"%~dp0jdk_1.7.0_55"
del "%~dp0jdk_1.7.0_55\tools.zip"
for /r "%~dp0jdk_1.7.0_55" %%x in (*.pack) do "%~dp0jdk_1.7.0_55\bin\unpack200" "%%x" "%%x.jar"
for /r "%~dp0jdk_1.7.0_55" %%y in (*.pack) do ren "%%y" "%%~ny.pack.org"
for /r "%~dp0jdk_1.7.0_55" %%z in (*.pack.jar) do ren "%%z" "%%~nz"
for /r "%~dp0jdk_1.7.0_55" %%a in (*.pack) do ren "%%a" "%%~na.jar"
for /r "%~dp0jdk_1.7.0_55" %%y in (*.pack.org) do del "%%y"

And lastly, double click "getJDK.bat" to start the process. After the process finished, the folder "jdk_1.7.0_55" should contain a workable JDK.

Don't forget to set the JAVA_HOME environment variable to point to your JDK folder and also add the %JAVA_HOME%\bin to the PATH environment in order to use the JDK wherever you are.

EcereSDK 0.44.09.0 Portable with OpenSSL (shared and static)

Finally have time to play with EcereSDK. In my own opinion, it is more convenience than Qt.

• Size is small
• Easier to build than Qt
• The SDK is more portable 

There already have portable versions around, the EcereSDKPortable by Ryoanji. However, it is based on 0.44 SDK two years ago without any new updates. So, I decided to build my own portable version for the latest 0.44.09.9 as of my writing. Instead of using Mingw-w64, I have chosen TDM-GCC.

Thanks to Ecere Teams' efforts, building the SDK on Windows platform is very easy and fast, no quirks needed. Just download the sdk source, the toolchains and upx. Open a command prompt with administrator right and set up the corresponding path. Finally run "mingw32-make" will do the job. As I am building a portable sdk, I did not run "mingw32-make install".

This only takes me around half an hour to build the sdk with a fully functional ide. And it only that me another half hour to build the static libraries as well. That's a big contrast with Qt.You may argue that Qt may be much more powerful, e.g. they have include QML, WebKit, JavaScript engine, ... and others. But the size and time to take for building Qt just not my cup of tea as I just want a simple tools to create cross-platform applications.

After building the SDK, it is time for me to really play with the eC language.

If you are interested to my build, you can find the final products and the corresponding tools at EcereSDK Portable.

• EcereSDKPortable-0.44.09.9_tdmgcc-4.7.1-3_openssl-1.0.0e.7z: Just extract the file (preserve folder structure) to any folder you want. After unzip, run "<FolderForExtraction>\EcereSDKPortable\ide.bat" to start using the tools.
• EcereSDKPortable-0.44.09.9_tdmgcc-4.7.1-3_openssl-1.0.0e_build_script.7z: This contains the Windows batch script, tools (7z.exe and wget.exe) and some supporting files. The batch file is handcrafted by myself and may contains errors not trapped. What the script doing is using wget to grab the files required to build the SDK from their original site (you can find the location inside the batch file with variable urlXXX. If you cannot find the files in the original web site, a copy of them can be found in the same folder as well. The script just build the x32 and x64 shared version of the SDK. Static libs and SQLCipher drivers (shared and static) required to use the IDE to build as I cannot find ways to just using command line to build all the stuff. If anyone has any idea on how to do all the stuff with Windows command line, please share with me so that I can perform some fully automated build testing stuff.

The batch file has only been tested on Windows 7 64 bit and Windows Server 2008 R2 64 bit platform as these are the only Windows boxes that is available to me. Building on 32 bit platform may not worked as the batch file assume default architecture is 64 bit. Tweaking with the batch file may help but not guaranteed. Running the batch inside VirtualBox will certainly failed, which may be due to the DirectX support is missing or incompatible.

2014/04/01
NOTE:  Due to unknown reason, in order to debug your project, you need to copy the gdb.exe as x86_64-w64-mingw32-gdb.exe under EcereSDKPortable\tdm\bin. It seems the ide cannot recognize the existence of gdb.exe (in version 0.44.08, I did not need to do this steps).

2014/04/02
NOTE: You will also need to download the gdb for x32 to debug the x32 version of your project. Save the gdb.exe as i686-w64-mingw32-gdb.exe under EcereSDKPortable\tdm\bin. This gdb.exe require libexpat-1.dll in the same folder, and jit-reader.h under EcereSDKPortable\tdm\include\gdb. 

Compile QT 5.2.0

Instructions to build Qt

This is my own experimented instructions for building Qt 5.2.0 shared (debug and release) and static (release only) libraries which can be used to compile programs using dll and run on Windows XP SP3.

Why build Qt by myself

The reason why I need to compile Qt by myself is that:

• Qt 5.2.0 official version cannot compile programs using dll that can be run on Windows XP SP3 as the tool chains not import some code from msvcrt.dll and causing exception when deployed on Windows XP SP3 (at least for my own program and dll)
• Qt Creator 3.0.0 official version requires installation of Microsoft VC Runtime, as it seems to be built by Microsoft Visual C++, I need a portable version, preferred not to install anything before used

Require files to be downloaded:

In order to build Qt by yourself, you need to download the followings:

• Qt5.2.0 (http://download.qt-project.org/official_releases/qt/5.2/5.2.0/single/qt-everywhere-opensource-src-5.2.0.zip)
• OpenSSL runtime (http://download.qt-project.org/development_releases/prebuilt/mingw_32/openssl-mingw-builds-runtime.7z)
• Python v2.7x or above (http://download.qt-project.org/development_releases/prebuilt/gdb/build-prerequisites/python.zip)
• MinGW (http://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win32/Personal%20Builds/mingw-builds/4.8.2/threads-posix/sjlj/i686-4.8.2-release-posix-sjlj-rt_v3-rev1.7z/download)
• Perl(v5.14 or above) (http://www.activestate.com/activeperl/downloads/thank-you?dl=http://downloads.activestate.com/ActivePerl/releases/5.16.3.1603/ActivePerl-5.16.3.1603-MSWin32-x86-296746.msi)

Building Environment

The environment that I used to build are as follow:

• Windows 7 Ultimate version
• UAC disabled (Otherwise, compilation may failed due to file access)
• Set Anti Virsus program to exclude the build directory and its sub-directory to speed up compilation (Also, some of the examples compiled will be identified as suspected malwares by say Symantec. If you are reluctant with this, add "-nomake examples" to the configure options).

Build directory structure

The directory structure that I used is as follow (originally want to have webkit built):
z:\QtBuild
    ├─5.2.0
    │    ├─mingw32
    │    └─mingw32-s
    ├─bin
    │    ├─mingw32
    │    └─QtCreator
    └─work
         ├─build
         │    ├─mingw32
         │    └─mingw32-s
         └─tools
                ├─mingw32
                ├─perl
                └─python

Preparation

• unzip MinGW (i686-4.8.2-release-posix-sjlj-rt_v3-rev1.7z) to z:\QtBuild\work\tools
• unzip MinGW (i686-4.8.2-release-posix-sjlj-rt_v3-rev1.7z) to z:\QtBuild\bin
• unzip python (python.zip) to z:\QtBuild\work\tools 
• install perl (ActivePerl-5.16.3.1603-MSWin32-x86-296746.msi) to z:\QtBuild\work\tools
• unzip OpenSSL runtime (openssl-mingw-builds-runtime.7z) to z:\QtBuild\5.2.0\mingw32 and z:\QtBuild\5.2.0\mingw32-s
• unzip qt-everywhere-opensource-src-5.2.0.zip to z:\QtBuild\work\build then rename folder "qt-everywhere-opensource-src-5.2.0" to "mingw32"
• unzip again qt-everywhere-opensource-src-5.2.0.zip to z:\QtBuild\work\build then rename folder "qt-everywhere-opensource-src-5.2.0" to "mingw32-s" (if you want to build the static version)
• unzip qt-creator-opensource-src-3.0.0.zip to z:\QtBuild\work\build then rename folder "qt-creator-opensource-src-3.0.0" to "qtc-3.0.0"
• copy libgcc_s_sjlj-1.dll, libstdc++-6.dll and libwinpthread-1.dll from z:\QtBuild\work\tools\mingw32\bin to z:\QtBuild\5.2.0\mingw32

(too long directory name may cause compilation failure)

Patching files for static build

Skip this if you don't need static build.

Open z:\QtBuild\work\build\mingw32-s\qtbase\mkspecs\win32-g++\qmake.conf with a text editor (e.g. Notepad)
Modify line 69 from

QMAKE_LFLAGS =

to

QMAKE_LFLAGS = -Wl,-Bstatic -lwinpthread -static -static-libgcc -static-libstdc++

and line 76 from

QMAKE_LFLAGS_DLL = -shared

to

QMAKE_LFLAGS_DLL = -Wl,-Bstatic -lwinpthread -static -static-libgcc -static-libstdc++

then save it.

Open z:\QtBuild\work\build\mingw32-s\qtbase\qmake\Makefile.win32 with a text editor
Modify line 50 from

LFLAGS        =

to

LFLAGS        = -static -static-libgcc -static-libstdc++

then save it.

Building Qt (shared lib debug-and-release version)

This will build Qt shared libraries with sqlite, odbc, openssl, opengl desktop. For details of the configure options, please check with (http://qt-project.org/doc/qt-5/configure-options.html) or issue "configure.bat -platform win32-g++ --help" in Windows Command Prompt.
Start a Windows Command Prompt and issue the following commands:

• set PATH="z:\QtBuild\work\build\5.2.0\mingw32\bin;z:\QtBuild\work\build\5.2.0\mingw32\qtbase\bin;z:\QtBuild\work\build\5.2.0\mingw32\qtrepotools\bin;z:\QtBuild\work\build\5.2.0\mingw32\gnuwin32\bin;z:\QtBuild\work\tools\mingw32\bin;z:\QtBuild\work\tools\mingw32\opt\bin;z:\QtBuild\work\tools\perl\bin;z:\QtBuild\work\tools\python;%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;"
• set "MAKE_COMMAND="
• set "QMAKESPEC="
• set "QTDIR=" set "DXSDK_DIR="
• cd /d z:\QtBuild\work\build\mingw32
• configure.bat -confirm-license -opensource -prefix "z:\QtBuild\5.2.0\mingw32" -shared -debug-and-release -platform win32-g++ -qt-sql-sqlite -qt-sql-odbc -D _CRT_SECURE_NO_WARNINGS -mp -no-vcproj -opengl desktop -no-angle -no-icu -skip webkit -openssl
• cd /d z:\QtBuild\work\build\mingw32
• mingw32-make

Building Qt (static lib release version)

This will build Qt shared libraries with sqlite, odbc, openssl, opengl desktop. For details of the configure options, please check with (http://qt-project.org/doc/qt-5/configure-options.html) or issue "configure.bat -platform win32-g++ --help" in Windows Command Prompt.
Start a Windows Command Prompt and issue the following commands:

• set PATH="z:\QtBuild\work\build\5.2.0\mingw32-s\bin;z:\QtBuild\work\build\5.2.0\mingw32-s\qtbase\bin;z:\QtBuild\work\build\5.2.0\mingw32-s\qtrepotools\bin;z:\QtBuild\work\build\5.2.0\mingw32-s\gnuwin32\bin;z:\QtBuild\work\tools\mingw32\bin;z:\QtBuild\work\tools\mingw32\opt\bin;z:\QtBuild\work\tools\perl\bin;z:\QtBuild\work\tools\python;%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;"
• set "MAKE_COMMAND="
• set "QMAKESPEC="
• set "QTDIR=" set "DXSDK_DIR="
• cd /d z:\QtBuild\work\build\mingw32-s
• configure.bat -confirm-license -opensource -prefix "z:\QtBuild\5.2.0\mingw32-s" -static -release -platform win32-g++ -qt-sql-sqlite -qt-sql-odbc -D _CRT_SECURE_NO_WARNINGS -mp -no-vcproj -opengl desktop -no-angle -no-icu -skip webkit -openssl -nomake examples -nomake tests -qt-libpng -qt-libjpeg -qt-pcre -qt-freetype -qt-zlib -ltcg
• cd /d z:\QtBuild\work\build\mingw32-s
• mingw32-make

QtCreator 3.0.0

I've not built QtCreator 3.0.0 by myself as I always got a debug build instead of release one. So I used one of the prebuilt one (http://sourceforge.net/projects/mingwbuilds/files/external-binary-packages/x32-Qt-5.2.0%2BQtCreator-3.0.0-%28gcc-4.8.2-dwarf%29.7z/download) I can find instead. The download also contain the qtbinpatcher.exe which is very useful for you to make your Qt tools set portable.

• unzip x32-Qt-5.2.0+QtCreator-3.0.0-(gcc-4.8.2-dwarf).7z to z:\QtBuild\bin and then rename the folder "Qt-5.2.0-i686" to QtCreator.

Make it Portable

To deploy the Qt and QtCreator, you only need the 5.2.0 and bin folders. To make this portable, create a batch file say QtCreatorPortable.bat under z:\QtBuild with the following contents:

@echo off
cls
if exist "%~dp05.2.0\mingw32\bin\qmake.exe" "%~dp0bin\QtCreator\qtbinpatcher.exe" --verbose --qt-dir="%~dp05.2.0\mingw32" --nobackup > NUL
if exist "%~dp05.2.0\mingw32-s\bin\qmake.exe" "%~dp0bin\QtCreator\qtbinpatcher.exe" --verbose --qt-dir="%~dp05.2.0\mingw32-s" --nobackup > NUL
if exist "%~dp0tools\QtCreator\bin\qmake.exe" "%~dp0bin\QtCreator\qtbinpatcher.exe" --verbose --qt-dir="%~dp0tools\QtCreator" --nobackup > NUL
start /b "" "%~dp0bin\qtCreator\bin\qtCreator.exe" -settingspath "%~dp0\bin\data"

NOTE: 

After starting QtCreator, you need to go to "tools -> options -> Build and Run" to change the settings in order to reflect the tool chains location.

OR

You could download my QtCreatorPortable.exe (https://drive.google.com/file/d/0B4DEiRGyVMoTT2VaLUFhTDJpVUU/edit?usp=sharing) and the initial settings data (https://drive.google.com/file/d/0B4DEiRGyVMoTQ0ktSkxFaTBUVU0/edit?usp=sharing) to ease the job. Unzip data.7z to z:\QtBuild\bin and save QtCreatorPortable.exe to z:\QtBuild. Just double the QtCreatorPortable.exe and it will use the qtbinpatcher.exe to patch the Qt libraries, update the settings and then start QtCreator. The assumption are as follow:

• Qt libraries are resided in .\5.2.0\mingw32 and/or .\5.2.0\mingw32-s
• QtCreator is resided in .\bin\QtCreator
• qtbinpatcher.exe is resided in .\bin\QtCreator
• QtCreator setting files in .\bin\data

ENJOY QT

References:

http://qt-project.org/wiki/MinGW-64-bit
http://qt-project.org/wiki/How-to-build-a-static-Qt-for-Windows-MinGW

Qt 5.2.0 released

Qt 5.2.0 has been released and so I will try compiling Qt 5.2.0 instead. To be honest, I have switched on the first day 5.2.0 released.
The problem that I encountered so far is, I cannot compiled with icu. Linker complained that it cannot find the library.
However, when I checked with my settings, the location of the library has already been added to environment path and libs. Cannot find the cause (by the way I use the same script to compile 5.1.1 without such issue).
Cannot link with icu means no webkit. That's a shame.
By the way, I have managed to compile without icu and will release the package to my partner to test it. The package will be a portable tools set (thanks qtbinpatcher author to make it possible) with Qt 5.2.0 shared libraries (debug and release) and static (release only). This tools set can compile executable using dlls that can be run on Windows XP SP3.
As promise to  +Alex Au, I will post the steps to compile Qt shortly.