file-243119101

Embedding Jython in Java Applications

jy_logo_large_c

Jython is an implementation of the Python programming language in Java, running on the Java Virtual Machine (JVM), as opposed to the standard implementation of Python which is implemented in C (CPython). Python developers can use it to take advantage of the vast libraries available to the Java community, which means adding a bit of Java integration in the application.

The reasons for running Jython instead of the default C implementation of Python or just sticking to Java when using the JVM differ from project to project. Personally, I have been using Jython to do rapid prototyping when Java implementations would have become too cumbersome, and to create flexible testbeds for Java applications which would be very hard to accomplish in Java. In addition, Jython can be a powerful way to add scripting capabilities to your Java application.

Getting started

First off, you need a JDK for the Java compiler and a copy of the Jython standalone JAR. There are also installers available for Jython, and it can be installed using many package managers; but for setting up your first development environment it’s a good idea to download the standalone binary to learn how everything fits together.

The simplest way to get the standalone JAR is to download it from Maven Central:

> curl -o jython.jar
http://search.maven.org/remotecontent?filepath=org/python/jython-standalone/2.7-b1/jython-standalone-2.7-b1.jar
 

That downloads the latest version, at the time of writing. If you use Maven for your builds you can of course add Jython directly to your dependencies. The important part is to have the JAR on your classpath when compiling and running your code.

Once you have access to javac and the Jython JAR file, you can create a minimal Java file to test embedding Jython:

import org.python.util.PythonInterpreter;
import org.python.core.*;
 
public class SimpleExample {
   public static void main(String []args) throws PyException
   {
       PythonInterpreter pi = new PythonInterpreter();
       pi.set(“integer”, new PyInteger(42));
       pi.exec(“square = integer*integer”);
       PyInteger square = (PyInteger)pi.get(“square”);
       System.out.println(“square: ” + square.asInt());
   }
}

We instantiate a PythonInterpreter object which we then use to access Jython functionality. Note that we use PyInteger objects when passing and retrieving data. This tell Jython to what data type our data should be converted.

This code also shows some of the ways to interact with the interpreter: set and get to define and get the value of variables, and exec to execute a statement, akin to entering it in the REPL (Read Eval Print Loop) and pressing Enter.

To run this code we need to compile it using javac:

> javac -cp jython.jar SimpleExample.java

 

And then we can run the resulting file using

> java cp jython.jar:. SimpleExample square: 1764
 

Shortcutting the Edit-Compile-Test Loop

With the Jython example above, you need to recompile the Java file every time you change the Jython code since it’s embedded in the Java source. That removes one of the benefits of Python over Java, namely the shorter feedback loop Python developers appreciate, because Python does not need to be compiled ahead-of-time.

To run the code without recompiling, you need to create a Python module, and use the PythonInterpreter to import it and run functions from it.

The Python module looks like any other Python module. I created a file called pymodule.py which implements a square() function:

def square(value):
   return value*value

This function can then be executed either by creating a string that executes it, or by retrieving a pointer to the function and calling its __call__ method with the correct parameters:

import org.python.util.PythonInterpreter;
import org.python.core.*;
 
public class ImportExample {
   public static void main(String [] args) throws PyException
   {
       PythonInterpreter pi = new PythonInterpreter();
       pi.exec(“from pymodule import square”);
       pi.set(“integer”, new PyInteger(42));
       pi.exec(“result = square(integer)”);
       pi.exec(“print(result)”);
       PyInteger result = (PyInteger)pi.get(“result”);
       System.out.println(“result: “+ result.asInt());
       PyFunction pf = (PyFunction)pi.get(“square”);
       System.out.println(pf.__call__(new PyInteger(5)));
   }
}

Now you can run the Java file, edit the Python file, and run it again without having to recompile (as long as the function signature does not change).

This has obvious benefits. You can now edit, run, re-edit, and re-run without rebuilding your project, even if you normally have a complicated tool-chain set up. It’s also helpful if you need to test changes on a target system where you cannot easily redeploy binaries. You can go farther with this, if even restarting a service is too expensive: Add a call to interpreter.exec(“reload(pymodule)”);somewhere where it can be triggered at will, allowing you to reload the module without restarting the system.

Duck-typing and Overloading

Even though the above method works, it is convoluted. It forces us to handle the complexities of translating between Java’s static typing and Python’s dynamic typing at the call site. Using Jython in this way may be enough if all you want is a quick test run, but for code that is intended to outlive the day, I strongly recommend using some kind of wrapping to make your Pythonic interface look like proper Java code. Non-ideomatic code is seldom a good idea, since any future reader of the code has to deal with the cognitive dissonance it causes, making understanding the code that much harder.

Using a wrapping class lets us use Java’s method overloading to hide the fact that we need to call our method using different types of arguments. For the square() function, we implement one version for integer values, and another for doubles; because Python has no double data type, we use PyFloat instead:

import org.python.util.PythonInterpreter;
import org.python.core.*;
 
public class CleanImportExample {
   public class PyModule {
       private PythonInterpreter interpreter;
       private PyFunction py_square;
 
       public PyModule() {
           this.interpreter = new PythonInterpreter();;
           this.interpreter.exec(“from pymodule import square”);
           this.py_square = (PyFunction)this.interpreter.get(“square”);
       }
 
       public int square(int val) {
           return py_square.__call__(new PyInteger(val)).asInt();
       }
 
       public double square(double val) {
           return py_square.__call__(new PyFloat(val)).asDouble();
       }
 
   }
 
   public void run() {
       PyModule module = new PyModule();
       System.out.println(module.square(2));
       System.out.println(module.square(2.2));
   }
 
   public static void main(String [] args) throws PyException {
       new CleanImportExample().run();
   }
 
}

Here, we instantiate the PythonInterpreter in the wrapper class. This means that the class’ users need to know nothing about Jython; but it also means that we may end up with several instances of the interpreter, which is probably not what we want.

Generally, you can use a singleton pattern to create your PythonInterpreter, as long as your Python modules are stateless. One way to implement this pattern is to have a class dedicated to holding a shared PythonInterpreter:

public final class SharedPythonInterpreter {
    public static final PythonInterpreter interpreter = new PythonInterpreter();
}

Explicitly naming it Shared makes it easier to remember not to manipulate state in ways that may break, and any class can now fetch an interpreter using SharedPythonInterpreter.interpreter. Of course, there are cases where you want to keep state in your interpreter; in those cases I recommend instantiating it in the wrapper constructor, or passing it into the constructor if it should be shared among several classes.

The final code, also adding crude namespacing to the import statement so that we won’t collide with any other imports named square, then looks like this:

import org.python.util.PythonInterpreter;
import org.python.core.*;
 
public class FinalExample {
   public class PyModule {
       private PythonInterpreter interpreter = SharedPythonInterpreter.interpreter;
       private PyFunction py_square;
 
       public PyModule() {
           this.interpreter.exec(“from pymodule import square as PyModuleSquare”);
           this.py_square = (PyFunction)this.interpreter.get(“PyModuleSquare”);
       }
 
       public int square(int val) {
           return py_square.__call__(new PyInteger(val)).asInt();
       }
 
       public double square(double val) {
           return py_square.__call__(new PyFloat(val)).asDouble();
       }
 
   }
 
   public void run() {
       PyModule module = new PyModule();
       System.out.println(module.square(2));
       System.out.println(module.square(2.2));
   }
 
   public static void main(String [] args) throws PyException {
       new FinalExample().run();
   }
}

Performance

Go into this with your eyes open. Jython performance suffers in comparison to pure Java code. Jython code also often suffers compared to CPython code, either because CPython spends a lot of time actually running C extensions (at C speeds) or because the memory consumption, as well as the startup time (mainly interesting for short-running programs) is significantly lower than for Java.

The one case where Jython may reliably outperform CPython is when doing purely CPU-bound tasks in a threaded environment, since the lack of a GIL on the JVM means that a computer’s cores may be fully utilized. Still, if performance is the main issue, Jython may just not be a very good fit.

Politics

In addition to performance, another word of warning may be in order: Introducing Jython into your Java stack is very easy, but you are still introducing a new programming language as well as new dependencies and increasing the complexity of your stack. In many cases doing this is fine, or even welcome. But to do it without signoff from the appropriate stakeholders is not likely to make you any friends, especially if you are working in a project or at a company that lacks strong Python knowledge.

Finding a new language added “under the radar” in a core product may very well be what turns someone against Python forever (“Python? That unmaintainable garbage XYZ snuck in?”) So spend a bit of time ensuring that other people on your team are on board with the idea and are prepared to take the effort to learn Python.

Final words

If you do have permission to use Jython, you just may start to see use cases everywhere;  there are a lot of things that would take hundreds of lines of Java that are trivial when you have access to Python and its standard library. I’ve found that to be the case especially when doing operating system integration or automation on Unix systems (a traditional strong point for Python), or doing low-level socket programming; using Jython allows for some dramatic simplifications and productivity gains. So go forth and make the Java world a bit more Pythonic!

See also:

cta-jeff-sutherland-webinarod2

subscribe-1

  • nEJC76

    The performance issue is mostly because Jython will re-complile python script/module every time it is imported.

    In my case for using Jython its bit differently than you show here – rather than use it as a provider of external methods, I use it as a main event processor.
    I have solved the performance issue with creating scripts that run in own thread/loop and are controlled via setting script variables from within java like:
    public void quitScript() {
    this.interpreter.set(“QUIT”, “True”);
    }

    You can also pre-compile python script and re-run whenever needed:

    String script = “print variable_1, variable_2″;
    PyCode compiled = org.python.core.__builtin__.compile( script, “”, “exec” );
    PythonInterpreter pi = new PythonInterpreter();
    // reuse the following code
    pi.set( “variable_1″, “value_1″);
    pi.set( “variable_x”, “value_x”);
    pi.exec( compiled )

    It gets even faster when JVM optimizations had a chance of running a couple of times thru the code – for me it runs almost with java speed.

    Your FinalExample should perform quite good also since it only compiles once.

  • Seth @ FBT

    Hey Fredrick, You have made some very interesting points here and I loved reading your instructions on how to embed Jython. We at http://www.fireboxtraining.com/ help anyone interested in learning python through tutorials and classes and your tutorial is very relevant to our work.

  • Dean

    Great post :)
    I have a question.
    javac -cp jython.jar SimpleExample.java Is this the only way to execute python embedded java code?

    I want to integrate python library to android app. Could you tell me there is any method to do this?