Tuesday, October 13, 2015

Open Visual Trace Route version 1.6.2

Open Visual Trace Route version 1.6.2 released.

Download at Sourceforge.net

Release note :Version 1.6.2 is out. Here is what's new:
- Add whois feature in both traceroute and sniffer
- Fix duplicated route points during the traceroute
- Fix command+Q failed to close the application on MacOSX
- Fix bug on sorting on tables
- Update Geoip database
- Update public ip address provider
- Fix a bunch of issues

Sunday, May 4, 2014

Open Visual Traceroute Demo video

Open Visual Trace Route version 1.5.0

Open Visual Trace Route version 1.5.0 released.

Download at Sourceforge.net

Release note :

* Add packet sniffer mode
* Add timeout for traceroute
* 2D map component using Openmap
* Upgrade to WorldWind 2.0
* Refactoring of code
* Use proper logger inside the code

Sunday, February 16, 2014

Workaround the sun.nio.cs.FastCharsetProvider bottleneck

The CharsetProvider is the class that, given a String representation of a Charset gives you the corresponding Charset object.
This is typically used when you do a new String(bytes[], "UTF-8") or a Charset.forName("UTF-8").
Looking closer at the Charset class tells us that it is actually using two levels of caches :

private static Charset lookup2(String charsetName) {
        Object[] a;
        if ((a = cache2) != null && charsetName.equals(a[0])) {
            cache2 = cache1;
            cache1 = a;
            return (Charset)a[1];

        Charset cs;
        if ((cs = standardProvider.charsetForName(charsetName)) != null ||
            (cs = lookupExtendedCharset(charsetName))           != null ||
            (cs = lookupViaProviders(charsetName))              != null)
            cache(charsetName, cs);
            return cs;

        /* Only need to check the name if we didn't find a charset for it */
        return null;

and if cannot find you charset in the cache, will use the standardProvider which is a sun.nio.cs.StandardCharsets that extends sun.nio.cs.FastCharsetProvider which implementation is synchronized as you can see :

public final Charset charsetForName(String charsetName) {
        synchronized (this) {
            return lookup(canonicalize(charsetName));

So if you are not lucky and uses more than two different encoding, you will go to this synchronized block and create a contention point in your application, as other people talked about herehere and also in this java ticket.

To prevent this issue from happening, we can directly use a Charset object since Java 1.6 in your  code. But regarding all the library that you are using, you will have a hard time patching all of them, as mentioned in this very good post.

Or, we could just patch Java at the source, and then use whatever version of the library and of java that we want, and apply this patch to old systems as well.

package sandbox;

import java.lang.reflect.Field;
import java.nio.charset.Charset;
import java.nio.charset.spi.CharsetProvider;
import java.util.HashMap;
import java.util.Iterator;
import java.util.Map;

import org.cliffc.high_scale_lib.NonBlockingHashMap;

import com.google.common.collect.ImmutableMap;

 * NonBlockingCharsetProvider to workaround the contention point on
 * {@link CharsetProvider#charsetForName(String)}
 * @author Leo Lewis
 * @see java.nio.charset.spi.CharsetProvider
 * @see java.nio.charset.Charset
public class NonBlockingCharsetProvider extends CharsetProvider {

 private CharsetProvider parent;

 private boolean lazyInit;

 private Map<String, Charset> cache;

  * @param parent
  *            parent charset provider
  * @param lazyInit
  *            if lazy init, init the cache when the application needs the
  *            charset, otherwise populate with the parent in the constructor
  *            if lazy init, will use a ConcurrentMap as it might be changed
  *            and iterated concurrently, otherwise, will use a
  *            guava Immutablehashmap
 public NonBlockingCharsetProvider(final CharsetProvider parent, final boolean lazyInit) {
  this.parent = parent;
  this.lazyInit = lazyInit;
  if (!lazyInit) {
   Map<String, Charset> tmp = new HashMap<>();
   Iterator<Charset> it = parent.charsets();
   while (it.hasNext()) {
    Charset charset = it.next();
    tmp.put(charset.name(), charset);
   cache = ImmutableMap.copyOf(tmp);
  } else {
   cache = new NonBlockingHashMap<>();

 public Charset charsetForName(final String name) {
  Charset charset = null;
  // if not lazyInit, the value should already be in the cache
  if (lazyInit && !cache.containsKey(name)) {
   // no lock here, so we might call several times the parent and put
   // the entry into the cache, it doesn't matter as the cache will be
   // populated eventually and we won't have to call the parent anymore
   charset = parent.charsetForName(name);
   cache.put(name, charset);
  return cache.get(name);

 public Iterator<Charset> charsets() {
  if (lazyInit) {
   return parent.charsets();
  return cache.values().iterator();

  * Save it if we want to reinstall, set up several times the provider
 private static CharsetProvider standardProvider;

  * Replace the CharsetProvider into the Charset class by an instance of this
  * {@link NonBlockingCharsetProvider}
  * @param lazyInit
  *            see
  *            {@link NonBlockingCharsetProvider#NonBlockingCharsetProvider(CharsetProvider, boolean)}
 public static void setUp(boolean lazyInit) throws Exception {
  Field field = Charset.class.getDeclaredField("standardProvider");
  if (standardProvider == null) {
   standardProvider = (CharsetProvider) field.get(null);
  NonBlockingCharsetProvider nonBlocking = new NonBlockingCharsetProvider(standardProvider,
  field.set(null, nonBlocking);

  * Restore the default java provider
  * @throws Exception
 public static void uninstall() throws Exception {
  if (standardProvider != null) {
   Field field = Charset.class.getDeclaredField("standardProvider");
   field.set(null, standardProvider);

Call the NonBlockingCharsetProvider.setUp(); to replace the java provider using reflection by this non blocking one. 
This provides two modes, lazy that will get the value from the parent when necessary and put it into a concurrent non blocking hashmap (better that the standard ConcurrentHashMap), and a non lazy that get all the parent values at initialization and provides them with a thread safe guava ImmutableHashMap. Performances are pretty close for both mode, the difference is if you want to duplicate the entire Charsets supported by the JRE into the cache, or just the one that your application is using.

Et voila!

Code source is on Github
Benchmark source as well

Monday, September 16, 2013

Open Visual Traceroute 1.4.0

Open Visual Trace Route version 1.4.0 released.

Download at Sourceforge.net

Release note :
* Allow to choose the network interface to use for trace route
* Optimize startup of traceroute 
* Dramatically improve DNS lookup performance 
* Add route length and distance between route points 
* Display Country flag in the table and in the map 
* Fix the overlay of labels on the map when several route points have the same coordinates 
* Redesign gantt chart view to make it easier to see points 
* Implement selection synchronization map<->table<->gantt chart 
* Change window layout 
* Change look and feel for windows os 
* Refactor code 
* Fix an issue when replaying a route with only one point 
* Fix Linux start script 
* Fix bug when saving the size of the window

Saturday, August 10, 2013

Open Visual Trace Route 1.3

Open Visual Trace Route version 1.3 released.

* Upgrade to Worldwind 1.5.0, change layer panel, add graticule 
* Add Gantt view of the route 
* Add Replay function of the traceroute 
* Implement bi-directional selection synchronization map<->table 
* Focus on the last point of the route during tracing 
* Update visual of the Route (3d shape) 
* Update labels of the points (cities) of the route 
* Save application window size and split location when exiting the application 
* Highlight current route point during tracing (both map and table) 
* Fix an error when clearing selection the table 
* Fix an error that crashed the application when starting from a directory that has space characters inside its path 
* Fix a memory leak when tracing the route

Order of Shutdown Hook executions

In a previous post I talked about restarting a Java application programmatically.

Since that, I used that code on a couple of Java applications, and found out that the code didn't work well when using some libraries, especially if these libraries register a shutdown hook.

Why is that so ? One of the trick used to restart was to execute the start command in a shutdown hook, to be sure that everything is cleaned up (especially socket servers or objects that are a little sensitive regarding the allocation of a port or a given exclusive resource) before restarting.

The issue is that this trick only works if these objects clean up themselves before the restart command is executed, which is not necessarily the case if this clean up is done in another shutdown hook (which is the case for Jetty web server for example).
The reason for that is that Shutdown hooks executions are not ordered.

To convince yourself, you can take a look at the java.lang.ApplicationShutdownHooks source and see that the collection of shutdown hook threads is stored in an IdentityMap, which is unfortunately not ordered.

I make a parenthesis here, because seeing that code made me wonder how was then managed the File.deleteOnExit(); case that, I believed naively being just another shutdown hook.
Actually, their are different types of shutdown hooks, with different priorities, and these priorities are managed by the java.lang.Shutdown class. This class is pretty peculiar one, and by looking at its code, we learn that their are actually 3 types of shutdown hooks :
  • One fo the Console, to restore the echo.
  • One for hooks registered by the application through the Runtime.getRuntime().addShutdownHook(Thread)
  • And one for the File.deleteOnExit()
And we also learn that the number of types of hook shall not exceed 10 (why not just 3 ?!?, we don't know)
private static final int MAX_SYSTEM_HOOKS = 10;
private static final Runnable[] hooks = new Runnable[MAX_SYSTEM_HOOKS];
End of the parenthesis. We now understand a little more how all these things connect together, and can think of a solution for my Shutdown hook, that I want to be sure that it will be executed after the other ones, without knowing who (what other 3rd part library) register shutdown hooks.
The solution that I came up with, was to execute all the hooks manually, clear the IdentityMap, add my hook and finally System.exit(). This will give something like that :
Where the executeShutdownHooksNow() method will use reflection to run the hooks manually :
private void runShutdownHooksNow() {
 try {
  Field hooksField = Class.forName(
  IdentityHashMap<Thread, Thread> currentHooks = (IdentityHashMap<Thread, Thread>) hooksField
  for (Thread thread : currentHooks.keySet()) {
   try {
   } catch (InterruptedException e) {
   } catch (Throwable e) {
    // just print, if executed within a strandart hook it would be ignored anyway
 } catch (Exception e) {
With this solution, we ensure that our hook will be run last.