HttpURLConnection and GET query parameters

I regret that JAVA SE still lacks any decent native HTTP client framework! Simple use cases like GET requests with query parameters require handcrafted coding with HttpURLConnection.

This article appeared first at techtavern.wordpress.com.

HttpURLConnection misses methods to set query parameters. There is no URI builder facility as found on Apache HTTP Client. You are on your own.

Therefore, I wrote the applyParameter method that takes care of GET query parameters.

URI baseUri = new URI("www.exemple.com/search");
URI uri = applyParameters(baseUri, "word","java");
HttpURLConnection connection = 
    (HttpURLConnection) uri.toURL().openConnection();
connection.setDoInput(true);
connection.setDoOutput(false);
connection.setRequestMethod("GET");
connection.connect();
if (connection.getResponseCode() == 
   HttpURLConnection.HTTP_OK) {
   // ...
}

The applyParameters method looks like:

URI applyParameters(URI baseUri, String[] urlParameters)
   StringBuilder query = new StringBuilder();
   boolean first = true;
   for (int i = 0; i < urlParameters.length; i += 2) {
     if (first) {
        first = false;
     } else {
        query.append("&");
     }
     try {
        query.append(urlParameters[i]).append("=")
             .append(URLEncoder.encode(urlParameters[i + 1], "UTF-8"));
     } catch (UnsupportedEncodingException ex) {
        /* As URLEncoder are always correct, this exception
         * should never be thrown. */
       throw new RuntimeException(ex);
     }
   }
   try {
      return new URI(uri.getScheme(), uri.getAuthority(), 
         uri.getPath(), query.toString(), null);
   } catch (URISyntaxException ex) {
     /* As baseUri and query are correct, this exception
      * should never be thrown. */
      throw new RuntimeException(ex);
   }
}

I had to bother about catching two exception that I know that will never be thrown: URISyntaxException and UnsupportedEncodingException. I had to join parameters by hand into a string. I had to care about safe web encoding. Shame on you JAVA!

Fortunately, at least I figured out that the URI constructor allows to append the query string to the base URI.

Compiling your abstract syntax tree transformation with Netbeans IDE

This article explains how to configure the Netbeans IDE to recognize java code the uses compiler internal classes. These classes are not part of the standard JDK and are not recognized by IDEs such as Netbeans, which will refuse to compile the correct code.

The compiler internal classes are used mainly as a hack to define annotations that automatically generate code. The most significant example is the Project Lombok, which transparently generates getters, setters and other code bload. Under the hood, the “Pluggable Annotation Processing API”  (JSR 269) provides a mechanism that exposes internal classes from the java compiler. These classes provide methods that allow us change the abstract syntax tree.

In order to configure Netbeans IDE to recognize the java compiler internal classes, one needs to add the them to the compiler class path. These classes are found within the lib/tools.jar library within the JDK 6 or 7. In order to add this library to the class path, the following procedure may work:

  1. Open the Netbeans IDE, open the Java Platform Manager (menu Tools->Java Platforms). The dialog will list, one the left side, one or more JDKs, one of them marked as default. On the right side, all jar from the compiler bootstrap classpath are listed. We need to add the lib/tools.jar into the list, which is not possible from the dialog.
  2. Click on “Add Platform” and follow the wizard to add the same default platform again, but using another name, for exemple, “JDK 1.7 – javac”.
  3. Close the dialog and the Netbeans IDE.
  4. Open the xml file: C:\Users\name\AppData\Roaming\NetBeans\7.4\config\Services\Platforms\org-netbeans-api-java-Platform\JDK_1.7_-_javac.xml. The path may vary according to your operating system, user name and Netbeans IDE version.
  5. Scan through the xml file for the “sun.boot.class.path” property.
  6. Add the full absolute path of the JDK lib/tools.jar file, don’t forget the “:” separator.
  7. Open the Netbeans IDE again, open the Java Platform Manager.
  8. Make sure that the right side lists the lib/tools.jar file.
  9. Close the dialog.
  10. Open your project properties, go to the libraries tab.
  11. Change the Java Platform field to “JDK 1.7 – javac”.

Some bibliography:

SymmetricDS syncronizing two tables with different names

While working with our newly discovered SymmetricDS tool, I realized that their manual does not clearly state how to synchronize two databases if their table names do not match. That might happen while matching tables between databases of two different companies, each one with its own table name policies. Blame those Database Administrators!

Here is how I accomplished synchronized two tables with different names:

First, I did the usual configuration: I defined two node groups A and B, defined a link between those node groups, defined a channel and a router for each direction (A to B  and B to A).

The description of triggers requires more configuration effort: Since the table names are different on each database, and since each trigger definition is bound to a table name, I had to define one trigger for each table name. Here, I suppose to databases: A and B, where the equivalent tables are called TableA and TableB.

insert into sym_trigger
   (trigger_id,source_table_name,channel_id,last_update_time,create_time)
values
   ('TriggerA','TableA','channel',current_timestamp,current_timestamp),
('TriggerB','TableB','channel',current_timestamp,current_timestamp);

As usual, the trigger needs to be associated with a router. Additional care is required: TriggerA is applicable only on database A and the reported changes are meaningful to be routed to database A to B. TriggerB is handled similarly.

insert into sym_trigger_router
   (trigger_id, router_id, initial_load_order, create_time, last_update_time)
values
   ('TriggerA', 'routerAtoB', 1, current_timestamp, current_timestamp),
('TriggerB', 'routerBtoA', 1, current_timestamp, current_timestamp);

Then, a table transformation must be defined: From node A to node B, table is to renamed from TableA to TableB. And from node B to node A, table is to renamed from TableB to TableA. The 'IMPLED' column policy means that all columns are to be copied without changes. I assume that both tables have the same column names and types. The delete action is ‘DEL_ROW’ and is not the main subject of this article. I prefered to run the transformation during the ‘Load’ transformation point.

insert into sym_transform_table
   (transform_id,source_node_group_id,target_node_group_id,transform_point,source_table_name,target_table_name,delete_action,column_policy)
values
   ('TransfAtoB', 'nodeA', 'nodeB', 'Load', 'TableA', 'TableB', 'DEL_ROW', 'IMPLIED'),
('TransfBtoA', 'nodeB', 'nodeA', 'Load', 'TableB', 'TableA', 'DEL_ROW', 'IMPLIED');

How SymmetricDS successfully replaced our complicated client-server architecture

With SymmetricDS , we avoided investing effort in designing, implementing and testing a complex, asynchronous communication protocol over web-service layer. Instead, we achieved a simple, intuitive interface that allowed us to deliver out solution within few days.

Our application started with a local standalone server that processes long running transactions. It naturally evolved to a distributed architecture. While benefits of the new architecture were clear to us and our customers, an issue arose : we would force our customers to enable their applications with boring and non intuitive web-services to communicate with our transaction server.

In short, the following requirements were considered:

  • The customer application creates data to be queued to our transaction server. Those data must be somehow uploaded to their respective transaction server instance that is running the queue.
  • Furthermore, if the customer updates data that was already sent to the server instance, then these changes must be reflected on the server, too (for shortness, I will omit details, but the transaction server handles transactions with input data that changes over time).
  • Finally, as soon as the transaction server instance concludes the transaction, the resulting status must be returned to the costumer application.

Since the transactions are long running, the requirements would probably resort to a complex, asynchronous communication protocol over web-service layer.

Motivated by the SymmetricDS features, we decided to adopt conventional database tables as communication a interface for our transaction server. The customer application is expected to insert and update data that shall be processed by the transaction server. The same table records the transaction status as soon as the transaction is complete. The transaction server contains identical tables as a queue of pending and completed tasks. SymmetricDS  does all the work of running the required web-services between customer and the transaction server to keep the tables consistent and synchronized.

I confess that such solution is really less fascinating than popular web-service/rest architectures. But the solutions was immediately understood by junior software analysts and inexperienced programmers. And allowed us to deliver the solution about 10 times faster than expected.

How to prevent EOFException when socket is closed

When one needs some very simple java client that reads some data from socket, then it might be a bit annoying to effectively detect that there is no more data to read, aka “End of File”. Here I show an approach to solve some issues to avoid spurious IOExceptions.
Read more of this post

Java: Class.forName() to retrieve inner class

Sounds trivial, but was not obvious because Eclipse IDE mislead me: How do I get a reference to a inner class by calling Class.forName() ?

package a.b.c;
public class Outer {
   ...
   public static class Inner {
      ...
   }
   ...
}

I first tried to call Class.forName() passing as parameter the fully qualified name generated by Eclipse:

Class innerClass = Class.forName("a.b.c.Outer.Inner");

But that was wrong! After wondering for some time, I found the (obvious solution):

Class innerClass = Class.forName("a.b.c.Outer$Inner");

Looks like as Eclipse “fully qualified” name is not exactly the same fully qualified name that the Java compiler uses to identify inner classes.

Serializing Hibernate objects as XML using XStream

People have been looking for  serializing Hibernate as XML using XStream, a task that is not straightforward.

I designed a solution that I would like to share with the open source community, called xstream-for-pojo.

Read more of this post