FPC JVM/Building

From Free Pascal wiki
Jump to navigationJump to search

Building the JVM compiler

The instructions below are for Unix-like platforms, but by changing the forward slashes to backslashes (and adding .exe after the compiler binary name when copying it), most, if not all, commands will work on Windows, too.

Every sequence of commands below starts from the assumption that you are in the top-level FPC source directory.

Prerequisites

A complete and working installation of FPC 2.6.0 or later (matching the version of the JVM compiler you want to build), the FPC JVM utilities, and an installation of the JDK 1.5 or later (with the binaries/jar files of all of them in the path).

Obtaining the FPC sources

The FPC JVM backend is now part of the main branch. 

 https://gitlab.com/freepascal.org/fpc/source/-/tree/main/compiler/jvm

To build the compiler, clone the repository to a local folder "fpc" from 

  https://gitlab.com/freepascal.org/fpc/source

If the latest official release of FPC, installed on your system does not match the version in the main branch, don't use it to build the JVM. Instead, check out the relevant tagged release from the repository!

Building the compiler and RTL

Note: all of the commands below require that the latest native FPC release for your platform is in the PATH.

Execute the following command in the top level directory of the checked out sources, i.e. in the fpc directory (replace FullPathToInstallLocation with the directory where to install the result): 

make CROSSOPT="-O2 -g" CPU_TARGET=jvm OS_TARGET=java INSTALL_PREFIX=FullPathToInstallLocation all crossinstall

If you wish to also create a run time library that can be used for Android development, use the following command (replace FullPathToInstallLocation with the directory where to install the result): 

make CROSSOPT="-O2 -g" CPU_TARGET=jvm OS_TARGET=android INSTALL_PREFIX=FullPathToInstallLocation all crossinstall

The generated compiler is called ppcjvm/ppcjvm.exe.

In both cases, a ppcjvm (or ppcjvm.exe) binary will be created and installed. The compiler generated by both build processes is the same, so you only need one. When compiling programs using this compiler, specify the correct -Fu parameter to point to the Java/Android units.

Building the fpcjres utility

This step is optional and is only required for the raw {$r xxx} support. Run the following command in the top-level directory of the checked-out sources: 

make all

The compiled fpcjres utility will be in utils/fpcres.

Additional tools

Building javapp

The tool used to generate Pascal import headers from Java classes is called javapp. It is written in Java because it is based on the javap utility from the JDK (version 6).

The source code for javapp can be found under utils/javapp in the FPC source tree. To build it, create a new Eclipse project and import the sources. Then, add jgrapht.jar from http://www.cs.sunysb.edu/~algorith/implement/jgrapht/distrib/ to the project (used for topological sorting to determine the dependencies between the classes). The main class file is Main.java.

Creating the Java RTL headers

To create the rtl/java/java_sysh.inc and rtl/java/java_sys.inc files, run javapp as follows: 

 java -jar javapp.jar -protected java.lang.System java.lang.Object java.io.Serializable java.io.IOException java.io.IIOException java.lang.Readable java.lang.Float java.lang.Double java.lang.Byte java.lang.Short java.lang.Integer java.lang.Long java.lang.Boolean java.lang.String java.lang.StringBuilder java.lang.StringBuffer java.lang.CharSequence java.lang.AbstractStringBuilder java.lang.Appendable java.lang.Number java.lang.IndexOutOfBoundsException java.lang.RuntimeException java.lang.IllegalArgumentException java.lang.IllegalStateException java.lang.UnsupportedOperationException java.lang.Exception java.lang.Throwable java.lang.NoSuchMethodException java.lang.LinkageError java.lang.Error java.lang.Comparable java.lang.Character java.text.Collator java.lang.Cloneable java.lang.Runtime java.lang.ThreadLocal java.math.BigInteger java.util.Comparator java.lang.reflect.Array java.lang.Class java.lang.reflect.GenericDeclaration java.lang.reflect.Type java.lang.reflect.Method java.lang.reflect.AccessibleObject java.lang.reflect.Member java.lang.reflect.AnnotatedElement java.lang.reflect.InvocationTargetException java.lang.reflect.Field java.util.Arrays java.util.Map java.util.HashMap java.util.Set java.util.Collection java.util.AbstractMap java.util.Calendar java.lang.Iterable java.lang.Enum  java.lang.Math java.util.AbstractCollection java.util.AbstractSet java.util.EnumSet java.util.BitSet java.util.Iterator java.nio.Buffer java.nio.ByteBuffer java.nio.CharBuffer java.nio.charset. -i -o java_sys

To create the rtl/java/jdk15.pas and rtl/java/jdk15.inc files, use 

 java -jar javapp.jar -x java.lang.System -x java.lang.Object -x java.io.Serializable -x java.io.IOException -x java.io.IIOException -x java.lang.Readable -x java.lang.Float -x java.lang.Double -x java.lang.Float -x java.lang.Double -x java.lang.Byte -x java.lang.Short -x java.lang.Integer -x java.lang.Long -x java.lang.Boolean -x java.lang.String -x java.lang.StringBuilder -x java.lang.StringBuffer -x java.lang.CharSequence -x java.lang.AbstractStringBuilder -x java.lang.Appendable -x java.lang.Number -x java.lang.IndexOutOfBoundsException -x java.lang.RuntimeException -x java.lang.IllegalArgumentException -x java.lang.IllegalStateException -x java.lang.UnsupportedOperationException -x java.lang.Exception -x java.lang.Throwable -x java.lang.LinkageError -x java.lang.Error -x java.lang.Comparable -x java.lang.Character  -x java.text.Collator -x java.lang.Cloneable -x java.lang.Runtime -x java.lang.ThreadLocal -x java.math.BigInteger -x java.util.Comparator -x java.lang.reflect.Array -x java.lang.Class -x java.lang.reflect.GenericDeclaration -x java.lang.reflect.Type -x java.lang.reflect.Method -x java.lang.reflect.AccessibleObject -x java.lang.reflect.Member -x java.lang.reflect.AnnotatedElement -x java.lang.reflect.Field -x java.lang.reflect.InvocationTargetException -x java.util.Arrays -x java.util.Map -x java.util.HashMap -x java.util.Set -x java.util.Collection -x java.util.AbstractMap -x java.util.Calendar -x java.lang.Iterable -x java.lang.Enum -x java.lang.Math -x java.util.AbstractCollection -x java.util.AbstractSet -x java.util.EnumSet -x java.util.BitSet -x java.util.Iterator -x java.nio.Buffer -x java.nio.ByteBuffer -x java.nio.CharBuffer -x java.nio.charset. -a sun. -a com.sun. -a apple. -a java.awt.Dialog -protected java. javax. org. -o jdk15

Creating the Android RTL headers

To create the rtl/android/jvm/java_sysh_android.inc and rtl/android/jvm/java_sys_android.inc files, use (same command line as for creating the equivalent Java RTL includes, except for the additional bootclasspath and extdirs parameters) 

 java -jar javapp.jar -bootclasspath /full/path/to/android-sdk/platforms/android-14/android.jar -extdirs . -protected java.lang.System java.lang.Object java.io.Serializable java.io.IOException java.io.IIOException java.lang.Readable java.lang.Float java.lang.Double java.lang.Byte java.lang.Short java.lang.Integer java.lang.Long java.lang.Boolean java.lang.String java.lang.StringBuilder java.lang.StringBuffer java.lang.CharSequence java.lang.AbstractStringBuilder java.lang.Appendable java.lang.Number java.lang.IndexOutOfBoundsException java.lang.RuntimeException java.lang.IllegalArgumentException java.lang.IllegalStateException java.lang.UnsupportedOperationException java.lang.Exception java.lang.Throwable java.lang.NoSuchMethodException java.lang.LinkageError java.lang.Error java.lang.Comparable java.lang.Character java.text.Collator java.lang.Cloneable java.lang.Runtime java.lang.ThreadLocal java.math.BigInteger java.util.Comparator java.lang.reflect.Array java.lang.Class java.lang.reflect.GenericDeclaration java.lang.reflect.Type java.lang.reflect.Method java.lang.reflect.AccessibleObject java.lang.reflect.Member java.lang.reflect.AnnotatedElement java.lang.reflect.InvocationTargetException java.lang.reflect.Field java.util.Arrays java.util.Map java.util.HashMap java.util.Set java.util.Collection java.util.AbstractMap java.util.Calendar java.lang.Iterable java.lang.Enum  java.lang.Math java.util.AbstractCollection java.util.AbstractSet java.util.EnumSet java.util.BitSet java.util.Iterator java.nio.Buffer java.nio.ByteBuffer java.nio.CharBuffer java.nio.charset. -i -o java_sys

To create the rtl/android/jvm/androidr14.pas and rtl/android/jvm/androidr14.inc files, use 

 java -jar javapp.jar -bootclasspath /full/path/to/android-sdk/platforms/android-14/android.jar -extdirs . -protected -x java.lang.System -x java.lang.Object -x java.io.Serializable -x java.io.IOException -x java.io.IIOException -x java.lang.Readable -x java.lang.Float -x java.lang.Double -x java.lang.Float -x java.lang.Double -x java.lang.Byte -x java.lang.Short -x java.lang.Integer -x java.lang.Long -x java.lang.Boolean -x java.lang.String -x java.lang.StringBuilder -x java.lang.StringBuffer -x java.lang.CharSequence -x java.lang.AbstractStringBuilder -x java.lang.Appendable -x java.lang.Number -x java.lang.IndexOutOfBoundsException -x java.lang.RuntimeException -x java.lang.IllegalArgumentException -x java.lang.IllegalStateException -x java.lang.UnsupportedOperationException -x java.lang.Exception -x java.lang.Throwable -x java.lang.LinkageError -x java.lang.Error -x java.lang.Comparable -x java.lang.Character  -x java.text.Collator -x java.lang.Cloneable -x java.lang.Runtime -x java.lang.ThreadLocal -x java.math.BigInteger -x java.util.Comparator -x java.lang.reflect.Array -x java.lang.Class -x java.lang.reflect.GenericDeclaration -x java.lang.reflect.Type -x java.lang.reflect.Method -x java.lang.reflect.AccessibleObject -x java.lang.reflect.Member -x java.lang.reflect.AnnotatedElement -x java.lang.reflect.Field -x java.lang.reflect.InvocationTargetException -x java.util.Arrays -x java.util.Map -x java.util.HashMap -x java.util.Set -x java.util.Collection -x java.util.AbstractMap -x java.util.Calendar -x java.lang.Iterable -x java.lang.Enum -x java.lang.Math -x java.util.AbstractCollection -x java.util.AbstractSet -x java.util.EnumSet -x java.util.BitSet -x java.util.Iterator -x java.nio.Buffer -x java.nio.ByteBuffer -x java.nio.CharBuffer -x java.nio.charset. -a sun. -a com.sun. -a apple. -a java.awt. java. javax. org. android. junit. -o androidr14

You will have to make several manual changes to the result before it compiles, though:

  • There is a circular dependency involving the AVViewGroup.LayoutParams nested class, which results in javapp being unable to order all classes. You will have to manually extract that class and redeclare it as AVViewGroup_LayoutParams = class external 'android.view' name 'ViewGroup$LayoutParams' (JLObject), and then replace all references to the original class to this new class
  • There is a class with a constant field called CREATE, which results in javapp generating wrong names for its constructors (<init>_). Rename the constant to CREATE_ and the constructors to Create
  • The declaration for AAActionBar.InnerTab is missing. Add it as InnerTab = class abstract external 'android.app' name 'Tab' end; inside AAActionBar

Creating interfaces for other Java classes

Example of creating a unit called bcel containing all classes under org.apache.bcel that can be found in bcel-6.0-SNAPSHOT.jar: 

 java -jar full/path/to/javapp.jar -classpath full/path/to/bcel-6.0-SNAPSHOT.jar -protected -o bcel org.apache.bcel.

The long parameter lists above are due to the split of the classes between the system unit and the jdk15 unit. The extra -classpath parameter is required when dumping classes that are not part of the standard JDK and has to point to the jar file(s) and/or directory/ies containing the class hierarchy you want to dump. If they are in the current directory, do not forget to pass explicitly"-classpath ."

Important:

  • If an identifier argument to javapp ends in a ".", it is interpreted as a package name, otherwise as a class name (not the '.' at the end of org.apache.bcel. in the example above).
  • In case circular dependencies of inner classes are involved, you must sacrifice one of the classes involved in the cycle by passing it using the -a option (anonymous). In that case, this class will be translated as a formal class (like java.awt.Dialog above, which is involved in a circular dependency with an inner class of java.awt.Window).
  • The -i option causes javapp to generate include files. Otherwise, it generates a standalone unit.
  • Pascal does not support the concept of Java-style inner classes, but only that of static nested classes. As a result, Java inner classes are not added to the translated headers.

The javapp source code is, unfortunately not very clean. The fact that the javap source code could have been cleaner did not help, but the situation has not exactly improved...

Building javapp from command-line

The installation and usage of Eclipse might not be desirable. You can compile javapp using JDK tools only. JDK is a prerequisite. Thus, you should have it installed anyway.

To compile, you must call javac and compile all .java files in "fpc\utils\javapp\src\fpc\tools\javapp" directory. As mentioned earlier, sources depend on jgrapht-jdk1.5.jar. Thus, the class path must be specified during compilation.

Once sources are compiled, all .class files must be gathered into a .jar file. The manifest file must declare the entry point class and the dependency on jgrapht.

Below is the Windows batch (.bat) script file that performs the compilation. The script is intended to be run from fpc\utils\javapp folder. Directory referencing JDK and jgrapht location should be adjusted according to your end system configuration. 

:: Set environmental variables.
:: If you have the JDK bin folder in your "PATH" environmental variables
:: you can remove the reference to jdkbindir at all
:: but make sure that all slash-separators are also removed
set jdkbindir=C:\Program Files\Java\jdk1.8.0_91\bin

:: This is the directory where jgrapht .jar file is located
:: make sure you remove the slash at the end 
set jgraphtdir=C:\jgrapht-0.7.2

:: This generates a small manifest file template
:: It's needed so, "jar" utility will create a dependency for jgrapht-jdk1.5 library
@echo Class-Path: jgrapht-jdk1.5.jar > manifest.txt

::: Compiling sources using Javac
:: -d .\ forces "javac" to put all the class files hierarchy  into the current directory
:: once "javac" finished you should have the following file structure created "./fpc/tools/javapp/"
:: The structure is important for .jar file
"%jdkbindir%\javac.exe" ^
  .\src\fpc\tools\javapp\*.java ^
  -d .\ ^
  -cp %jgraphtdir%\jgrapht-jdk1.5.jar

:: Creating a .jar file for the application
::  Including manifest file (manifest.txt)
::  Specifying entry point  (fpc.tools.javapp.Main)
::  "fpc" is the folder where .class files are located
"%jdkbindir%\jar" cfme javapp.jar manifest.txt fpc.tools.javapp.Main fpc

copy /Y "%jgraphtdir%\jgrapht-jdk1.5.jar" jgrapht-jdk1.5.jar

:: Clean up time!
:: We don't need this file anymore
:: /Q is quite mode
@del /Q manifest.txt
:: Remove the directory structure containing .classes. We don't need them since they should now be part
:: of the .jar file. Removing it, quietly 
@rmdir /S /Q fpc

As the result, you'll have two files javapp.jar and jgrapht-jdk1.5.jar. If you need to move javapp.jar to another directory, copy jgrapht-jdk1.5.jar as well.

External tools

Building Jasmin

Jasmin is the compiler's Java byte code assembler to translate the generated assembler files (*.j) into Java class files (*.class).

Building BCEL

BCEL is a Java byte code verification framework that can be used to look for errors in Java class files. See FPC_JVM/Debugging for usage information.

  • Check out the svn sources using the following command
 svn checkout http://svn.apache.org/repos/asf/commons/sandbox/classscan/trunk/bcel bcel
  • Before building, apply the patch at ftp://ftp.freepascal.org/pub/fpc/contrib/jvm/bcel-r1157681-returntype.patch to prevent incorrect verification errors about return type mismatches.
  • To build BCEL, you first have to install Apache Maven
  • Once Maven is installed and its mvn binary is in your PATH, run mvn package in the bcel source directory. The first time you do so, Maven will download and configure many plugins, which takes quite some time. After building, the new version of BCEL will be available as target/bcel-6.0-SNAPSHOT.jar
Retrieved from "https://wiki.freepascal.org/index.php?title=FPC_JVM/Building&oldid=157221"