O Java EE 6 Application Servers, Where Art Thou?

Nearly two years ago (time flies), when Java EE 6 came out, I wrote a post about application servers where I did some micro benchmarking (basically, startup time). I had plenty of comments and recently I had many people asking for some updates. Witht Java EE 6 booming, with some cloud vendors moving to Java EE 6, it was time to update this microbenchmark and focus on Java EE 6 application servers. BTW, if you want to know what Java EE 6 is, you can check the slides of a presentationI gave a few times.

The benchmark

Same disclaimer as last time : This is not a real benchmark ! so I’ll copy paste the paragraph I wrote last time :

In this test I’m just concerned about the usability of an application server for a developer. The idea is to download it, install it, start it and take some measurements : size of download, ease of installation, size of disk, startup time, size of RAM... That’s all. No deployment of an application, no fancy twists to gain performance… Because some of these application servers are resource consuming, I’m doing all my tests on a Windows XP SP3 virtual machine (running on Virtual Box 4.1). It is a fresh install of Windows XP sp3 with 1Gb of RAM and not too many software installed. So when I boot there are 27 processes running and using 230 Mb of RAM (leaving 770 Mb free). Virtualizing can be slower, so keep in mind that startup time can be a bit faster that what I’m giving you here. I use JDK 1.6.0_27 (when it’s not bundled with the server). No optimization at all is made (I haven’t twisted the JVM, or antyhing application server parameter, everything comes out of the box).

To calculate the startup time, I don’t do any fancy rocket science either. I just start the server a few times, check how long it takes and use the best startup time. Also remember that some servers do not load any container at startup, making them very fast to start. That’s why I first trigger the web admin console (when there’s one) so I’m sure at least one web application is deployed (I did a print screen of the admin console when there’s one). Then I just stop and start the server a few times and get the best startup time (by reading the logs). To calculate the memory footprint, I just use the Windows task manager and check the size of the java.exe process.

Which application servers ?

If you look at the Java EE Compatibility page at Oracle, you’ll see that there are a bunch of application servers that actually have passed the Java E E6 TCK (Oracle GlassFish Server 3.1.1Caucho Resin 4.0.17JBoss Application Server 7Apache TomEE 1.0.0-beta-1,  TMAX JEUS 7, IBM WebSphere Application Server 8.0Fujitsu Interstage Application Server) and some still haven’t (Geronimo 3, JOnAS 5.3, Siwpas). On this list some application servers implement the full Java EE 6 and others just implement the Web Profile 1.0.

GlassFish 3.1.1

GlassFish 3 is the open source reference implementation for Java EE 6. The 3.1 release has added clustering capabilities. Being the reference implementation, GlassFish 3.x implements both Java EE 6 and the Web Profile 1.0, that’s why you have two different bundles you can download (one of 80.6 Mb and the other 45.8 Mb).

Version 3.1.1
What do you get ? Java EE 6 & Web Profile 1.0
Who is behind? Oracle
Open source? Dual license CDDL and GPL
Documentation Documentation
Download Zipfile, executables
Size of the downloaded file 80.6 Mb (Java EE 6) 45.8 Mb (Web Profile)
Installation mode Unzip or exe
Size once installed on drive 96.1 Mb (Java EE 6) 57.8 Mb (Web Profile)
How to start it Execute %GLASSFISH_HOME%\bin\asadmin start-domain
Location of log files %GLASSFISH_HOME%\domains\domain1\logs
Best startup time 2 953ms (Java EE 6) 1 914ms (Web Profile)
Size of the process in RAM at startup 83 MB (Java EE 6) 69 MB (Web Profile)
Welcome page http://localhost:8080/
Admin console http://localhost:4848

Resin 4.0.23

Resin 4 was one of the first web server (such as Jetty or Tomcat) to move to the Web Profile 1.0. For me it’s the success story that confirmed that profiles in Java EE were needed. It’s really difficult to go from a Web Server to a full Java EE 6 application server (remember that there’s still Entity CMPs in EE 6) but much easier to implement the Web Profile. Resin is very CDI centric and based on the Caucho implementation called CanDI.

Version 4.0.23
What do you get ? Web Profile 1.0
Who is behind? Caucho
Open source? GPL License
Documentation Documentation
Download Zip, tgz, deb
Size of the downloaded file 24.2 Mb
Installation mode Unzip the file
Size once installed on drive 32 Mb
How to start it Execute %RESIN_HOME%\java -jar lib/resin.jar start
Location of log files %RESIN_HOME%\log
Best startup time 1 845ms
Size of the process in RAM at startup 2 processes : 68.5 MB + 39 MB
Welcome page http://localhost:8080/
Admin console http://localhost:8080/resin-admin/

JBoss 7.0.2-Final

It looks like JBoss 7 is, finally, a nice millésime. For those of you who waited so long for JBoss 5 to get certified and then didn’t see any JBoss 6, JBoss 7 is what you need with a fantastic startup time compared to the older versions (in my previous benchmardJBoss 5 was the slowest app servers, slower than WebSphere 7). Another novelty is an admin console (sexier than the good old JMX Console).

Version 7.0.2-Final
What do you get ? Web Profile 1.0
Who is behind? JBoss/RedHat
Open source? LGPL
Documentation Documentation
Download Zipfile
Size of the downloaded file 70.7 Mb
Installation mode Unzip
Size once installed on drive 77.9 Mb
How to start it Execute %JBOSS_HOME%\bin\standalone
Location of log files %JBOSS_HOME%\standalone\log
Best startup time 2 782ms
Size of the process in RAM at startup 70.7 MB
Welcome page http://localhost:8080/
JMX console http://localhost:9990/console

TomEE 1.0.0-beta-1

Like Resin, Apache TomEEis the perfect success story for the Web Profile. TomEE is no more than Tomcat + OpenWebBeans + OpenEJB + OpenJPA + MyFaces + other bits. It really shows that Java EE is a jigsaw puzzle where you can take open standards, bundle them together and become a certified Web Profile application server.

Version 1.0.0-beta-1
What do you get ? Web Profile 1.0
Who is behind? Apache
Open source? Apache Software License
Documentation Documentation
Download Zip, tar.gz
Size of the downloaded file 24.2 Mb
Installation mode Unzip the file
Size once installed on drive 29.2 Mb
How to start it Execute %TOMEE_HOME%\bin\startup.bat
Location of log files %TOMEE_HOME%\logs
Best startup time 3 288ms
Size of the process in RAM at startup 65 MB
Welcome page http://localhost:8080/
Admin console (*) http://localhost:8080/manager/html

(*) Looks like to be able to log on to the admin console, you need to change the %TOMCAT_HOME%\conf\tomcat-users.xml configuration files and add a manager-script role to a user. But I couldn’t make it work.

Geronimo v3.0-M1

Geronimo is not already Java EE 6 ceritified (still in Miletone 1). Geronimo comes in 2 flavours, one with Jetty the other one with Tomcat. And like TomEE, Geronimo bundles several open source implementationsof Java EE.

Version 3.0-M1
What do you get ? Not yet certified
Who is behind? Apache
Open source? Apache Software License
Documentation Documentation
Download Zip, tar
Size of the downloaded file 66.9 Mb (Geronimo + Tomcat)
Installation mode Unzip the file
Size once installed on drive 76.4 Mb
How to start it %GERONIMO_HOME%\bin\geronimo start
Location of log files %GERONIMO_HOME%\var\log
Best startup time 12 467ms
Size of the process in RAM at startup 159 Mb
Welcome page http://localhost:8080/
Admin console (*) http://localhost:8080/console-base

(*) user/pwd : system/manager

JOnAS 5.3.0-M4

JOnASis till not Java EE 6 certified. It provides a full Java EE 5 execution stack plus some Java EE 6 previews (JPA 2.0, CDI, …), supporting EJB3 (and not EJB 3.1 yet).

Version 5.3.0-M4
What do you get ? Not yet certified
Who is behind? OW2
Open source? LGPL
Documentation Documentation
Download Zip, tar.gz
Size of the downloaded file 211 Mb
Installation mode Unzip the file
Size once installed on drive 231 Mb
How to start it %JONAS_HOME%\bin\jonas start
Location of log files %JONAS_HOME%\logs
Best startup time 11 036ms
Size of the process in RAM at startup 107 Mb
Welcome page http://localhost:9000/
Admin console (*) http://localhost:9000/jonasAdmin

(*) user/pwd : jonas/jonas

Siwpas Enterprise 2.0.0-GA

Siwpas (Simple Web Profile Application Server) is quite new compared to the good old application servers. But like most of them, it embeds well known open source stacks, most from Apache  (Tomcat, OpenEJB, OpenWebBeans, MyFaces, OpenJPA, Apache BeanValidation). It’s open source but you’ll still need a 30 days trial license to startup the enterprise version of the server.

Version 2.0.0-GA
What do you get ? Web Profile 1.0 (Not yet certified)
Who is behind? Mechsoft
Open source?
Documentation Documentation
Download Zip, exe, tar.gz
Size of the downloaded file 34.2 Mb
Installation mode Unzip the file
Size once installed on drive 38.2 Mb
How to start it %SIWPAS_HOME%\bin\startup
Location of log files %SIWPAS_HOME%\logs
Best startup time 8 598ms
Size of the process in RAM at startup 72.8 Mb
Welcome page http://localhost:8080/
Admin console (*) http://localhost:8080/console

(*) user/pwd : admin/admin

Websphere 8.0.0.1

Websphere 8 is the Java EE 6 application server of IBM. Like the previous versions, you need a big bandwith to download one gigabyte of modules that get installed in nearly 2 hours. I noticed that there is a developer edition but once installed it looks like the 2.2 Gb of the full server (I might got lost in the maze of the IBM website). There is also a Community Edition and it looks like it’s based on Geronimo (and the executable to install it is also 116 Mb). IBM, please, make it simple.

Thanks to @sebsto I found a gem : it looks like IBM is working on a version WAS 8.5 Alpha that would look like the other app servers (light and easy to install). I’ll stay tuned on this one.

Version 8
What do you get ? Java EE 6
Who is behind? IBM
Open source? Commercial product
Documentation Documentation
Download Exe file for WAS & WAS for developers
Size of the downloaded file 116 Mb
Installation mode Execute + Wizard install (with all the defaults) which downloads more stuff (took nearly 1:40 hour)
Size once installed on drive 2.2 Gb
How to start it Execute %WEBSPHERE_HOME%\AppServer\profiles\AppSrv01\bin\startServer.bat server1 -profileName AppSrv01
Location of log files %WEBSPHERE_HOME%\AppServer\profiles\AppSrv01\logs\server1
Best startup time 42 s
Size of the process in RAM at startup 155 Mb
Admin console https://localhost:9043/ibm/console

Jeus 7

A bit unknown but in version 7.0 (so it shows this application server has been around for long time), Jeus 7from TMaxSoft was a bit difficult to startup (create an account on the web site, download, install, run a wizard and read the README.txt to make it started). Then I discovered that there is no Jeus 7 for Windows so I downloaded the Linux version and installed it on Ubuntu (so the results of the microbenchmark are not very accurate compare to the other made on Windows).

Version 7
What do you get ? Java EE 6
Who is behind? TmaxSoft
Open source? Commercial
Documentation Documentation
Download bin file (only available on Linux)
Size of the downloaded file 126 Mb
Installation mode Execute the file and answer a wizard
Size once installed on drive 198 Mb
How to start it Execute $JEUS_HOME/bin/jeus -u administrator -p <pwd you declared in the wizard>
Location of log files $JEUS_HOME/domains/jeus-domain1/logs
Best startup time 23 783ms (but again that was on Ubuntu, not in Windows)
Size of the process in RAM at startup 147 Mb
Welcome page http://localhost:8088/ (but looks like there’s no page)
Admin console Not sure there’s a admin console, but jeusadmin tool helps you to manage the server

Interstage

I couldn’t even find where to click and download the server, so if anybody knows, I’ll be ready to give it a try.

Where the hell is Tomcat ?

Tomcat 7 is not a Java EE 6 application server nor does it implement the Web Profile 1.0. It is just a Servlet 3.0 container, so it’s shouldn’t appear in this blog post. But, everybody likes Tomcat, everybody feels Tomcat is way much faster than any server on earth (even faster than Neutrinos or maybe not). So I had to put it in this benchmark just to be a source of comparaison. And, indeed, Tomcat is the fastest application server of the list (3 times faster than GlassFish or 4 times faster than JBoss). But it only does Servlet 3 while the others implement between 12 (Web Profile) or 28 (Java EE 6) specifications.

Version 7.0.22
What do you get ? Servlet 3
Who is behind? Apache
Open source? Apache Software License
Documentation Documentation Index
Download Zip, tar.gz
Size of the downloaded file 7.5 Mb
Installation mode Unzip the file
Size once installed on drive 11.3 Mb
How to start it Execute %TOMCAT_HOME%\bin\startup.bat
Location of log files %TOMCAT_HOME%\logs
Best startup time 694 ms
Size of the process in RAM at startup 34.6 Mb
Welcome page http://localhost:8080/
Admin console http://localhost:8080/manager/html

Summary

As a summary I will show you two graphs with startup time and resource consumption so it’s easier to compare.

Startup time

When you are a developer, startup up time matters (it doesn’t when you are in production). Thanks to tools like JRebel or the hot deployment of our IDEs, we tend to restart our app servers less and less. But, sometimes it happens (PermGens, classpath mess up, memory leaks, refreshing app server cache…). I didn’t put WAS 8 nor Jeus 7 in the graph below because they would just break the scale (42 and 23 seconds). It’s interesting to notice that 4 app servers (Resin, GlassFish, JBoss and TomEE) start in less that 5 seconds (Jonas & Geronimo are still in Milestone). Resin is 3 times slower than Tomcat, but again, it comes with the Web Profile.

Size once installed on the drive & size of process in RAM

The size of the application installed on your disk is not very accurate : some app servers come with documentation, samples… others no. But still, that gives you an overview of how big an app server can be (again, I couldn’t put WAS in this diagram as it will break the scale with its 2.2 Gb). The memory consumption is more interesting. Without any JVM or server twists, when you startup, they nearly all take less that 100Mb of RAM(see how close TomEE, GlassFish, JBoss and Siwpas are).

Conclusion

First of all, as you can see there is no Weblogic. Unfortunatelly Weblogic is the last big player to be certified Java EE 6. I don’t know the roadmap, hopefully I’ll soon be able to update this blog entry with the latest version of Weblogic. Second, this is not a real benchmark : I didn’t deploy any application nor I did some stress tests. One task I would like to do is to deploy a Java EE 6 application on all these servers and see how portable the code is. But that’s for another blog.

What did I want to show in this blog ? Well, that application servers have changed. Except for WAS, Weblogic or Jeus, all the others are fast at startup and take few resources (memory).

And why did I want to show this ? Because I’m fed up to still hear in 2011 : “Tomcat is light, Java EE app servers are not”. As you can see, in terms of startup and memory, the gap between Tomcat and Resin, GlassFish or JBoss is really small.

How is it going to affect your development life ? Do you remember when 7/8 years ago we had to mock our database accesses because we didn’t want to install a fat oracle database on our laptop ? These days are over now because we use in-memory databases such as H2 or Derby. Same thing is happening with application servers : soon app servers will be running in memory and we won’t be mocking anything. We will fire a test case, app servers and databases in memory, and that’s it. And with tools like Arquilian, it looks like this future is already happening.

Long live Java EE 6 and Java EE 6 application servers !

Comments
23 Responses to “O Java EE 6 Application Servers, Where Art Thou?”
  1. rmannibucau says:

    thanks for this,

    jsut a “detail” note: for tomee the webadmin could be http://localhost:8080/openejb/ too

    Do you plan to do an embedded test? (i wonder where could be tomee-embedded)

    • agoncal says:

      Thanks. I hope I’ll have time to deploy an application (Web Profile) on all of these app servers, and why not use the EJB embedded container to do some unit tests maybe

  2. Nice article!

    even the not certified ones, had you the chance to test JAX-RS on ecah server? that could produce another excellent article.

    • agoncal says:

      The application I would like to deploy would follow the Web Profile (which doesn’t have JAX-RS), but I’ll try to include it.

  3. Good post, great work! I think startup times are very important in development. There has been a lot of improvement over the past years. I wish it could have been a priority in the WL8 days :).

    I’ve run similar measurements about a year ago ( http://dow.ngra.de/2010/12/14/jee-oss-container-startup-times-apples-vs-oranges/ ). There also we thought that next step is to actually deploy an application and measure the startup time then. Deployment will trigger a longer startup time because more services are used and will give a better picture. I wish there was a good app that could be used on all app servers (or maybe even on servlet containers) not just app server specific admin consoles.

  4. and please don’t use the task manager to check mem consumption but rather do it via yourkit profiler or jvisualvm plz – txs!

    LieGrue,
    strub

  5. Scott says:

    The other problem with including tomcat in the comparison is that it is not running any applications, where the appservers are all (except Jeus 7) running (and have accessed) a fairly complex web application (their console). The startup and memory times for tomcat are not really comparable in that sense either.

  6. KCiesielski says:

    I had the same problem with TomEE: could not create a user with admin console access. I figured out that there are some incosistencies between tomee documentation and example XML’s. A correct entry in tomcat-users.xml goes like:

    Two important things here:

    1. the “username” keyword (instead of “name” which can be found in examples)
    2. The “manager-gui” role

    Anyway, I found out that TomEE cannot run a simple servlet without web.xml, which is incorrect… Sad thing which makes me doubt in all the certification processes like web profile cert given to TomEE…

    • Bare with us on TomEE. The TCK uses JSR-88 for deployment and it’s become clear that this is just different enough from Tomcat’s drop-and-wait webapps dir approach to cause issues. We’re hacking up a second JSR-88 impl for the drop-and-wait approach and that will help us work out these issues. All should be well before TomEE goes final.

  7. KCiesielski says:

    My last comment has missing XML which should be (without left and right border):

    user username=”admin” password=”” roles=”standard,manager-script,manager-gui”

  8. rmannibucau says:

    @KCiesielski: TomEE allows servlet 3 deployment without a web.xml on trunk, did you try the SNAPSHOT version?

  9. KCiesielski says:

    Well, probably they fixed it. I tried version available on website (1.0.0-beta-1). It was the same website that stated loudly that TomEE was JEE6-WebProfile-certified, hence my disappointment.

  10. Martijn Blankestijn says:

    Well, just saw your post while reading the announcement of the launch of Weblogic 12.

    Interested to see how it will compare after the long long wait.

  11. gothmog says:

    Using latest weblogic 12c, i got really good startup times between 5 seconds and 7 seconds with an empty domain.

  12. Chase says:

    Time for a WebLogic and TomEE update!

  13. zhang ye says:

    Yes, please. update with WebLogic 12c and new JBoss 7.1 and GlassFish 3.1.2 would be great!

  14. lgarulli says:

    Hi,
    what about plain Jetty?

  15. WebSphere now has beta of the Liberty profile (to be shipped in WAS 8.5 in a couple of months) and it starts in under 3 seconds. You can test drive it now: http://wasdev.net/

  16. Michael says:

    This is sort of useless data. I really don’t care how big the executable is, or how much RAM it takes or how fast it is to startup or how “easy” or “difficult” the installer is. Once a server is up, it stays up for a LONG time. Often times, when a server stars up it runs a bunch of initialization code, warms-up caches, loads data from persistent storage, initializes state, etc… That stuff take WAY longer than the server actually starting up and the data you hold in RAM dwarfs the actual base server cost. So, I really don’t understand the point of this exercise, it provides no really relevant data for a real-world scenario.

    What is relevant is the performance characteristics of the various servers in terms of load, caching, messaging, DB management, etc… I want to know how does Glassfish do vs. Geronimo when we throw a couple hundred requests at it per second with a complex distribution in terms of various services, DB access, logging, event passing, etc…

    • Couldn’t agree more Micheal

    • henk53 says:

      Micheal, I don’t agree with you at all.

      What you’re asking matters, don’t get me wrong, but there are already benchmarks for that. SPEC for instance, you’ve probably heard about them.

      Those benchmarks are needed when you are a (very) large enterprise that has a multi-million line application, serving an established base of demanding customers and that needs to deal with a lot of traffic and gives out SLA guarantees, QoS, etc.

      For small shops, indy developers, people new to the platform trying out things, and I dare even say developers in general, the stuff Antonio tests here is of the *utter most importance*!

      In development, not even with JRebel, my server DOES NOT stay up for a LONG time. I continuously restart and redeploy. The size of the executable matters A LOT when you have different workspaces in say Eclipse, and assign each workspace an individual server to keep things simple. If all that’s needed to “install” a new server is to unpack a 25MB tarball (TomEE), then the ease with which I can create new workspaces for some radical experiments is unparalleled (and yes, developers need to experiment a lot).

      WebSphere here is atrocious. As a simple developer trying out things, I don’t care about uptimes, or requests per second. All I need is a Java EE runtime, so I can deploy my sample code and see how it behaves. Having to download gigabytes of data and sit through installs that take hours, is just undoable. From an operations point of view, where you installed WebSphere 6 in 2004 and let it sit there untouched for 10+ years, it might indeed not matter, but for an agile and dynamic developer it’s a total disaster. And I’m not even talking about the IBM JDK requirement and having to run both Eclipse and WebSphere with that JDK (which thus means I can’t run on OS X).

      Exactly your idea, that difficult installs and gigabytes of enterprise cr*p don’t matter, is EXACTLY what has been driving so many developers to alternative platforms. If you can’t grab the mindshare of new developers, then eventually your platform will just die out. It’s as simple as that.

      Luckily, not everyone thinks as you, since GlassFish, JBoss AS, TomEE, etc have been addressing exactly this requirement. Even former heavyweight WebLogic is almost there (it still requires an install instead of just an unpack, but they’re trying). And surprisingly, even IBM is getting the message as Liberty is exactly the kind of lightweight setup developers grave for. Unfortunately Liberty doesn’t support all Java EE APIs yet, but if it eventually does (even if only the web profile) IBM might have a winner there again.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 7,479 other followers

%d bloggers like this: