Why isn't it possible to set a timeout to avoid Nagios forcefully terminating the plugin with a 'Service Check Timeout' error?

Unfortunately, there is no straightforward way to set timeouts on JMX connections. Rumours are that this will be implemented in JMX 2.0 or Java 7, respectively.

Error opening RMI connection: Connection refused to host:

This happens when the /etc/hosts file on your remote server contains something like

       HOSTNAME, localhost

where HOSTNAME is the name of your server. You have to make sure that HOSTNAME isn't resolved as so change your /etc/hosts as follows:

          IP_ADDRESS      HOSTNAME

where IP_ADDRESS is the physical IP address of your remote server and HOSTNAME is its host name. These changes require that you restart your Java application.

Is it possible to define a regex threshold for triggering the threshold when the regex matches?

Yes, just prefix your regex with the @ sign. This will trigger the threshold if the value matches the regex threshold.

How can I trigger a threshold if a value falls below a given threshold?

Use threshold ranges as defined here: Threshold and Ranges

Does Syabru Nagios JMX plugin work with NRPE?

Contrary to some postings on the web, Syabru Nagios JMX plugin works perfectly with NRPE given that Java is installed on the remote host and that you got your NRPE setup right.

Is it possible to listen to JMX notifications and submit asynchronous passive checks in Nagios?

Check out Syabru Nagios Message Broker which acts as an intermediary between Nagios and JMX notifications.