<![CDATA[Not obvious ]]>https://pdudits.github.iohttps://cloud.githubusercontent.com/assets/1588543/21372338/c07f1c2e-c716-11e6-8daa-aa3b35a2d2cf.jpgNot obvious https://pdudits.github.ioRSS for NodeThu, 30 Nov 2017 15:11:04 GMT60<![CDATA[Log directly to Logstash from Payara]]>
Also published on Payara blog

When running multiple instances of an application server it is quite hard to see correlations between events. One of the best tools to enable that is the ELK stack - Elasticsearch for building fulltext index of the log entries, Logstash for managing the inflow the events, and Kibana as a user interface on top of that.

Solutions for Payara Server exist, that use better parseable log format]which can be then processed by Logstash Filebeat in order to have these log entries processed by a remote Logstash server.

In our project, we chose a different path — we replaced all logging in the server and our applications with Logback, and make use of the logback-logstash-appender to push the events directly to Logstash over a TCP socket. The appender uses LMAX disruptor internally to push the logs, so the processes does not block the application flow.

This article will show you how to have this configured for your project as well.

Log everything via Logstash

In first step, we replace the backend for Payara Server’s logging with logstash.

Project goodees was started by me to collect useful small components for Java EE, and Payara Server in particular. The first released component is payara-logback, that includes all libraries properly packaged to use logback as the logging backend in Payara Server and also includes an installation script.

Follow the instructions there to switch your logging.

Logback and logstash configuration

Our logback configuration looks similar to following:

<configuration scan="true">
    <jmxConfigurator />
    <shutdownHook/>
    <!-- propagate changes to java.util.logging --> (1)
    <contextListener class="ch.qos.logback.classic.jul.LevelChangePropagator">
        <resetJUL>true</resetJUL>
    </contextListener>

    <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        (2)
        <file>${LOG_DIR}/glassfish/${instanceName}/${instanceName}.log</file>
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <!-- daily rollover -->
            <fileNamePattern>${LOG_DIR}/glassfish/${instanceName}/${instanceName}.%d{yyyy-MM-dd}.%i.log</fileNamePattern>
            <timeBasedFileNamingAndTriggeringPolicy
                class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
                <maxFileSize>50MB</maxFileSize>
            </timeBasedFileNamingAndTriggeringPolicy>
            <maxHistory>60</maxHistory>
        </rollingPolicy>

        <encoder>
            <pattern>[%d{ISO8601}] [%t] [%4p] [%logger]: %m%n</pattern>
        </encoder>
    </appender>

  <appender name="json" class="ch.qos.logback.core.rolling.RollingFileAppender">(3)
    <file>${LOG_DIR}/glassfish/${isntanceName}/json/${isntanceName}.json</file>
    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
      <fileNamePattern>${LOG_DIR}/glassfish/${instanceName}/json/${instanceName}.%d{yyyy-MM-dd}.json.zip</fileNamePattern>
      <maxHistory>120</maxHistory>
    </rollingPolicy>
    <encoder class="net.logstash.logback.encoder.LogstashEncoder">
      <customFields>{"instance":"${instanceName}","type":"glassfish"}</customFields>
    </encoder>
  </appender>

  <appender name="logstash-socket" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
    <destination>${LOGSTASH_DEST}</destination>(4)
    <secondaryConnectionTTL>5 minutes</secondaryConnectionTTL>
    <encoder class="net.logstash.logback.encoder.LogstashEncoder">
      <customFields>{"instance":"${instanceName}","type":"glassfish"}</customFields>
    </encoder>
  </appender>

    <root>
        <level value="INFO" />
        <appender-ref ref="FILE" />
        <appender-ref ref="json"/>
        <appender-ref ref="logstash-socket"/>
    </root>
</configuration>

The following points should be noted:

1 Propagation between slf4j and `java.util.logging is needed
2 We put the logging directory outside of the domain directory, and controlled by an environment variable. instanceName is a JVM argument, that is unique for every instance in our system.
3 We keep the JSON formatted logs on disk with greater retention period than our Elasticsearch does. If we need to, we can re-upload those to analyze past events.
4 Mutliple destinations in form of host:port[,host:port] can be defined for the Logstash socket appender. We keep them in an environment variable.

We have similar configuration for logback-access.xml:

<configuration scan="true">
  <jmxConfigurator/>
  <shutdownHook/>
  <!-- always a good activate OnConsoleStatusListener -->
  <statusListener class="ch.qos.logback.core.status.OnErrorConsoleStatusListener" />

  <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>${LOG_DIR}/glassfish/${instanceName}/${instanceName}.access.log</file>
    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
      <fileNamePattern>${LOG_DIR}/glassfish/${instanceName}/${instanceName}.access.%d{yyyy-MM-dd}.log.zip</fileNamePattern>
      <maxHistory>90</maxHistory>
    </rollingPolicy>

    <encoder>
      <pattern>combined</pattern>
    </encoder>
  </appender>
  <appender-ref ref="FILE" />

  <appender name="json" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>${LOG_DIR}/glassfish/${instanceName}/json/${instanceName}.access.json</file>
    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
      <fileNamePattern>${LOG_DIR}/glassfish/${instanceName}/json/${instanceName}.access.%d{yyyy-MM-dd}.json.zip</fileNamePattern>
      <maxHistory>10</maxHistory>
    </rollingPolicy>
    <encoder class="net.logstash.logback.encoder.AccessEventCompositeJsonEncoder">
      <providers>
         <timestamp/>
         <version/>
         <method/>
         <protocol/>
         <statusCode/>
         <requestedUri/>
         <remoteHost/>
         <hostname/>
         <remoteUser/>
         <contentLength/>
         <elapsedTime/>
         <context/>
         <pattern><pattern>
           {"instance": "${instanceName}",
            "forwarded_for": "%header{X-Forwarded-For}",
            "server_name": "%v",
            "type": "access"}
         </pattern></pattern>
      </providers>
    </encoder>
  </appender>

  <appender name="logstash-socket" class="net.logstash.logback.appender.LogstashAccessTcpSocketAppender">
     <destination>${LOGSTASH_DEST}</destination>
     <secondaryConnectionTTL>5 minutes</secondaryConnectionTTL>
     <encoder class="net.logstash.logback.encoder.AccessEventCompositeJsonEncoder">
      <providers>
         <timestamp/>
         <version/>
         <method/>
         <protocol/>
         <statusCode/>
         <requestedUri/>
         <remoteHost/>
         <hostname/>
         <remoteUser/>
         <contentLength/>
         <elapsedTime/>
         <context/>
         <pattern><pattern>
           {"instance": "${instanceName}",
            "server_name": "%v",
            "forwarded_for": "%header{X-Forwarded-For}",
            "type": "access"}
         </pattern></pattern>
      </providers>
    </encoder>
  </appender>

  <appender-ref ref="logstash-socket" />
  <appender-ref ref="json" />
</configuration>

Finally, the logstash configuration.

It is quite a simple, however access log events tend to have field names that are incompatible with recent versions of elasticsearch. To work around this problem, they need to be adapted a bit, as shown in the filter below:

input {
  tcp {
    port => 1065
    codec => json_lines
  }
}

filter {
  if [type] == "access" {
     mutate {
        rename => {
          "@fields.HOSTNAME" => "HOSTNAME"
          "@fields.content_length" => "content_length"
          "@fields.elapsed_time" => "elapsed_time"
          "@fields.method" => "method"
          "@fields.protocol" => "protocol"
          "@fields.remote_host" => "remote_host"
          "@fields.remote_user" => "remote_user"
          "@fields.requested_uri" => "requested_uri"
          "@fields.status_code" => "status_code"
        }
     }
     if [forwarded_for] != "-" {
        mutate {
          rename => {
             "forwarded_for" => "remote_host"
          }
        }
     }
     mutate {
       remove_field => ["forwarded_for"]
     }
   }
}

output {
  elasticsearch  {
    hosts => #....
  }
}

Adapt your application

Your application also needs small tuning now to prevent various class incompatibility errors. Both slf4j-api, and logback-classic should now be provided dependencies of your application. In other words, they should not be included in your .war or .ear build.

Downsides

It is important to note that asadmin set-log-levels no longer works, as logback is the one filtering the levels. You will need a different way of controlling log levels with this kind of configuration.

Give feedback

Let us know how this setup works for you in comments on Payara blog. You’re also invited to share your projects' goodees!

]]>https://pdudits.github.io/2017/11/30/Log-directly-to-Logstash-from-Payara.htmlhttps://pdudits.github.io/2017/11/30/Log-directly-to-Logstash-from-Payara.html<![CDATA[Semi-automatic Jenkins upgrade on Windows]]>

Jenkins offers nice auto-update functionality, where it will download new version, schedules a restart and updates. But what to do when it is unable to download the version?

In our company there is this really annoying proxy antivirus scanner, that likes to inspect all jar and war files for viruses, so things like Wildfly or Netbeans take hours to download. That’s really bad new when one wants to update Jenkins, as it will time out after minute or two.

The usual approach I take for plugins that fail to download, is that I download them manually (e. g. by replacing http with https in the mirror link, then the antivirus is bypassed ;)) and upload them into Jenkins. This doesn’t work for the main war file, of course.

Should I’ve been running on Linux, I would just download the file, and overwrite existing file. I’m on Windows and the process have the file locked. I would really like just to trigger this logic for upgrading the file at startup.

To find out how that works I looked into Jenkins' WindowsLifecycleService:

    /**
     * On Windows, jenkins.war is locked, so we place a new version under a special name,
     * which is picked up by the service wrapper upon restart.
     */
    @Override
    public void rewriteHudsonWar(File by) throws IOException {
        File dest = getHudsonWar();
        // this should be impossible given the canRewriteHudsonWar method,
        // but let's be defensive
        if(dest==null)  throw new IOException("jenkins.war location is not known.");

        // backing up the old jenkins.war before its lost due to upgrading
        // unless we are trying to rewrite jenkins.war by a backup itself
        File bak = new File(dest.getPath() + ".bak");
        if (!by.equals(bak))
            FileUtils.copyFile(dest, bak);

        String baseName = dest.getName();
        baseName = baseName.substring(0,baseName.indexOf('.'));

        File baseDir = getBaseDir();
        File copyFiles = new File(baseDir,baseName+".copies");

        try (FileWriter w = new FileWriter(copyFiles, true)) {
            w.write(by.getAbsolutePath() + '>' + getHudsonWar().getAbsolutePath() + '\n');
        }
    }

So I did the same thing manually:

  1. Downloaded new war file to d:\temp

  2. Created file jenkins.copies in jenkins' home with content

    d:/temp/jenkins.war>d:/jenkins/jenkins.war

    I saved it with UNIX line endings and verified, that there is indeed just single \n at the end (service will fail to start if that is not the case).

  3. Had jenkins restart once it is idle

  4. DONE. Jenkins did update just as if it had downloaded the file itself.

I kept the .copies file handy for future updates. I will just put new version in temp and copy the .copies file over.

]]>https://pdudits.github.io/2017/09/07/Semi-automatic-Jenkins-upgrade-on-Windows.htmlhttps://pdudits.github.io/2017/09/07/Semi-automatic-Jenkins-upgrade-on-Windows.htmlThu, 07 Sep 2017 00:00:00 GMT<![CDATA[Your Java EE App on Kubernetes]]>

In previous post we managed to run Payara on Kubernetes. This is of course bit dull without applications deployed. Next we will create an actual aplication, that will deploy with official Payara container. We will use the full profile Payara, just to demonstrate, that microservices can be built using standard distribution.

Let’s see what we will need to have our app in Kubernetes as well:

  • .war Maven project

  • Fabric8 Maven plugin

  • Docker for Windows / Docker tools

Installing Docker tooling

Given my hobby environment (Windows 10 Home), I cannot use the current Docker for Windows installation — remember, there’s no HyperV in Home edition. We need to install Docker Toolbox, its retired ancestor.

Sidecar approach

I assume you don’t upgrade your application server as often as you upgrade your applications. But when you do upgrade your server, you shouldn’t need to rebuild all of your applications. At least I wouldn’t like to do that.

Kubernetes gives you solution to that, called Sidecar containers. Instead of running single image, you put multiple images together, and they can speak to each other via localhost, or share directories.

So we will take Official Payara Docker Image, mount its autodeploy directory, and for our app we will build an image based on busybox, that will just copy our .war file to that folder.

Kubernetes%20sidecar.png

The app-sidecar will be based on busybox, adding just 1MB of binaries to our app package.

Creating Docker images with Maven

One doesn’t need to leave its trusted tools for this fancy new tech! Everything you need to do concerning Docker images and Kubernetes you can achieve with Fabric8 Maven Plugin. Fabric8 is entire CI/CD pipeline for Kubernetes, and I will surely look into it in next posts — first, I need to understand what am I doing before I let magical tools do that for me ;).

The goal build of Fabric8 plugin constructs the Docker image based on image configuration. So this is what the interesting parts of pom.xml look like:

<plugin>
    <groupId>io.fabric8</groupId>
    <artifactId>fabric8-maven-plugin</artifactId>
    <version>3.4.1</version>
    <configuration>
        <images>
            <image>
                <name>pdudits/%a:%l</name> (1)
                <alias>app-sidecar</alias> (2)
                <build>
                    <from>busybox:1</from> (3)
                    <assembly>             (4)
                        <descriptorRef>artifact</descriptorRef>
                    </assembly>
                </build>
            </image>
        </images>
    </configuration>
    <executions>
        <execution>
            <id>fabric8-default</id>
            <goals>
                <goal>resource</goal>
                <goal>build</goal>
            </goals>
        </execution>
    </executions>
</plugin>
1 The name of docker image. %a stands for artifactId, %l stands for release verson or latest for snapshot
2 we can use this alias for defining kuberenetes resources later
3 Our base image
4 The Maven Assembly Descriptor Name, describing what should be copied into the image (inside directory /maven by default).

To actually run the build you need to have Docker environment configured, as the plugin will call out to Docker. Minikube has this nice feature, that it will reuse Docker daemon from within minikube VM. You therefore doesn’t need to run another VM, just make sure to follow instructions of minikube docker-env before invoking maven:

d:\src\k8se9s\payara-sidecar>minikube docker-env
SET DOCKER_TLS_VERIFY=1
SET DOCKER_HOST=tcp://192.168.99.100:2376
SET DOCKER_CERT_PATH=d:\src\kubernetes\.minikube\certs
SET DOCKER_API_VERSION=1.23
REM Run this command to configure your shell:
REM @FOR /f "tokens=*" %i IN ('minikube docker-env') DO @%i

d:\src\k8se9s\payara-sidecar>@FOR /f "tokens=*" %i IN ('minikube docker-env') DO @%i

That also means, that whatever you build is immediately available for deployment into your Kubernetes cluster.

Kubernetes Resource Descriptors with Maven

fabric8:resource will take YAML files that serve as templates for resource descriptors, add some useful labels to them, and attaches them as build results.

For our sidecar we will create a Deployment. It is a declaration of what Pod should look like with added features like replication — more than one instances of same pod — and rolling upgrade. Especially the rolling upgrade is interesting for us. When you upgrade your app, a new Pod will be created, and when it starts successfully, the trafic will be directed to it, and after that the old pod is stopped.

So let’s see the file:

spec:
  replicas: 1
  template:
    metadata:
      labels:
        run: payara
    spec:
      volumes:
      - name: apps                    (1)
        emptyDir: {}
      initContainers:
      - alias: app-sidecar            (2)
        volumeMounts:
        - name: apps                  (3)
          mountPath: /apps
        command: ["sh","-c", "cp -a /maven/*.war /apps"] (4)
      containers:
      - image: payara/server-full:171 (5)
        name: payara
        command: ["bin/asadmin", "start-domain", "-v"]
        ports:                        (6)
        - containerPort: 8080
          name: http
        - containerPort: 4848
          name: admin
        volumeMounts:
        - name: apps                  (7)
          mountPath: /opt/payara41/glassfish/domains/domain1/autodeploy
1 That’s the directory shared between Docker images
2 We refer to the image via alias from pom.xml
3 The shared volume is accessible via /apps for container app-sidecar
4 We copy the files. Every container need to have running process, otherwise it is marked as crashed. Therefore we just hang, waiting for input from the void.
5 Payara is the other container in the pod
6 Yes that’s the old version, you’ll soon find out why…​
7 And the files we copied into /apps Payara sees in its autodeploy directory.

Another resource desciptor we’ll add will describe a service. This will give a set of pods that offer same service a common, load balanced IP address inside the cluster, and may expose it to outside world.

In our minikube, it will expose the port of the VM. In the cloud it will integrate with provider’s load balancers.

spec:
  ports:
  - port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    run: payara
  type: NodePort

The service exposes port 8080 (the http port) as port 8080 of the service API. The calls will be directed to pods, that have label run equal to payara. And to expose it to outside world, we will use a port of the node Kubernetes runs on — the minikube VM in our case.

Deploying to the cluster

Now that we have run our mvn clean install, we’ll have our docker image pushed, and our resources created.

Next we’ll deploy with mvn fabric8:deploy.

[INFO] <<< fabric8-maven-plugin:3.4.1:deploy (default-cli) < install @ payara-sidecar <<<
[INFO]
[INFO] --- fabric8-maven-plugin:3.4.1:deploy (default-cli) @ payara-sidecar ---
[INFO] F8: Using Kubernetes at https://192.168.99.100:8443/ in namespace default with manifest d:\src\k8se9s\payara-sidecar\target\classes\META-INF\fabric8\kubernetes.yml
[INFO] Using namespace: default
[INFO] Updating a Service from kubernetes.yml
[INFO] Updated Service: \payara-sidecar\target\fabric8\applyJson\default\service-payara-sidecar.json
[INFO] Updating Deployment from kubernetes.yml
[INFO] Updated Deployment: \payara-sidecar\target\fabric8\applyJson\default\deployment-payara.json
[INFO] F8: HINT: Use the command `kubectl get pods -w` to watch your pods start up

Here we are!

27405216 fca5ba9c 56d0 11e7 9ed1 d77db9216400.png

Our .war that were deploying to Payara now deploys to docker container with payara server inside Kubernetes!

minikube service payara-sidecar

Will open browser with the app. You need to add /payara-sidecar to the path.

P. S.

Look what this does:

>kubectl set image deployment/payara payara=payara/server-full:172
deployment "payara" image updated
27405642 aaa048aa 56d2 11e7 9df9 350547abcdca.png

New pod was started with newer version of application server, and after it started, the old one was removed.

]]>https://pdudits.github.io/2017/06/22/Your-Java-EE-App-on-Kubernetes.htmlhttps://pdudits.github.io/2017/06/22/Your-Java-EE-App-on-Kubernetes.htmlThu, 22 Jun 2017 00:00:00 GMT<![CDATA[Running Payara in Kubernetes]]>

In the first part of hopefully longer series I will focus at the tasks one should do to get plain server — without the app, but a Java EE full profile one — up and running in Kubernetes cluster.

Installing the tools

For this series I will be using my Windows 10 Home notebook. Particulary important consequence of that is, that Hyper-V is not availble. I will therefore use VirtualBox.

Step 1: Install VirtualBox

There’s not much to be said to this one, just head over to VirtualBox download page, download and install.

Step 2: Install Kubernetes

Kubernetes is a tool for (virtual machine) clusters. However, you can run it all in single virtual machine on your computer! To achieve that you’ll need:

  • minikube: the management tool for the vm. Download minikube-windows-amd64.exe, put it in a dedicated folder for your kubernetes tools, and rename it to minikube.exe

  • kubectl: CLI tool for interacting with Kubernetes. They have fancy bash & curl script for downloading the current release, I’m pretty sure it’s almost of no use for you, the Windows guy. Here’s bit of PowerShell instead:

    $(
  $base = "https://storage.googleapis.com/kubernetes-release/release";
  $ver = iwr "$base/stable.txt" | % { $_.Content.Trim() };
  "$base/$ver"
) | % {
  iwr "$_/bin/windows/amd64/kubectl.exe" -OutFile kubectl.exe
}

Create and run the cluster

We’re just few commands away from running the cluster now.

Step 3: Prepare Environment vars

On typical home machine, you don’t need to set any variables. However, on "enterprise" one you might consider:

:: We will download some ISOs now
SET https_proxy=http://proxy.company.local:8080
:: Into directory %MINIKUBE_HOME%\.minikube, defaults to %HOMEPATH%.
:: Also a vm disk image gets created there
:: it also fails if you're not running minikube on your HOMEDRIVE
SET MINIKUBE_HOME=d:\src\kubernetes

Step 4: Create cluster

d:\src\kubernetes>minikube start --vm-driver=virtualbox
Starting local Kubernetes v1.6.0 cluster...
Starting VM...
Downloading Minikube ISO
 89.51 MB / 89.51 MB [==============================================] 100.00% 0s
SSH-ing files into VM...
Setting up certs...
Starting cluster components...
Connecting to cluster...
Setting up kubeconfig...
Kubectl is now configured to use the cluster.

You can see virtual machine minikube running in VirtualBox now. Let’s check, that it indeed works:

d:\src\kubernetes>kubectl cluster-info
Kubernetes master is running at https://192.168.99.100:8443

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

d:\src\kubernetes>minikube dashboard
Opening kubernetes dashboard in default browser...
c2301036 400c 11e7 8a3f 7c1a249a8624.png

Deploy Payara

Step 5: Create deployment

Bare deployment of Payara inside cluster can be achieved with

> kubectl run payara --image=payara/server-full --port=8080 --replicas 1
deployment "payara" created

This starts download of a docker image and creates a Deployment and a Replica Set, that in turn contains single Pod, that exposes standard HTTP port. It will take a while, since the docker image needs to be downloaded. You can monitor the progress like this:

> kubectl get events
LASTSEEN   FIRSTSEEN   COUNT     NAME                      KIND         SUBOBJECT                 TYPE      REASON              SOURCE                  MESSAGE
6m         6m          1         payara-4223602339-vr5w5   Pod                                    Normal    Scheduled           default-scheduler       Successfully assigned payara-4223602339-vr5w5 to minikube
6m         6m          1         payara-4223602339-vr5w5   Pod          spec.containers{payara}   Normal    Pulling             kubelet, minikube       pulling image "payara/server-full"
6m         6m          1         payara-4223602339         ReplicaSet                             Normal    SuccessfulCreate    replicaset-controller   Created pod: payara-4223602339-vr5w5
6m         6m          1         payara                    Deployment                             Normal    ScalingReplicaSet   deployment-controller   Scaled up replica set payara-4223602339 to 1

Alternatively you can use minikube dashboard for more interactive view.

Step 6: Expose HTTP port

After the replica set is running, you yet need to make it accessible from within container inside minikube VM to your host:

> kubectl expose deployment payara --port=80 --type=NodePort
service "payara" exposed
> minikube service payara
Opening kubernetes service default/payara in default browser...
e607bc24 40c6 11e7 80b7 a4ea7b632e3a.png

Six steps are already enough for a nice evening spent at your computer, in next part of the series we’ll try to draw the rest of the owl:

  • Deploying the app into server, ideally without every build result being 800MB

  • Improving logging

  • Customizing your domain

  • …​

Stay tuned! 

]]>https://pdudits.github.io/2017/05/23/Running-Payara-in-Kubernetes.htmlhttps://pdudits.github.io/2017/05/23/Running-Payara-in-Kubernetes.htmlTue, 23 May 2017 00:00:00 GMT<![CDATA[Setting up Angular2 CLI with Maven in enterprise network]]>

Working with Angular2 CLI is easy - you just wait for npm to download the Internet and afterwards and then ng will take care of all your tooling setup.

Not so much if at your workplace you’re running Windows, your network has NTLM-based firewall and your project is actually Java-based one built with Maven.

Even if the proxy for NPM can be configured via npm set or http_proxy environment variable, npm does not support no_proxy clause, so the things would start failing if you want to consume your own packages.

Needed infrastructure

Instead, we will use Nexus 3. We will install it on a server, that has reasonable access to Internet (e. g. no auth, or service user autorization to proxy), and it will do the proxying for all things NodeJS and NPM we need.

For running NPM as part of our Maven build, we’ll use Frontend Maven Plugin. It will take care of installing node, npm and project dependencies for us, and also for our CI build.

Maven module structure

We will lay out our modules in following way:

parent (pom)
\_ app-backend (war module)
\_ app-frontend (war module)
    \_ node (js project)

Module app-frontend will invoke npm build in directory app-frontend/node. I tried to be Maven compatible here at first, but it wasn’t really helping when trying to open the JS project in the IDE. Maven will then pack the result of npm into a .war file.

In case you want to deploy single artifact, app-backend may depend on app-frontend and the build result will be then copied to backend .war.

Setting up proxies in Nexus

After you install nexus and verify that it is able to download artifacts you will need following repositories.

Repo Name Type Remote path Note

npm

npm (Proxy)

https://registry.npmjs.org

We actually use group, along with our release repository, but this is minimal setup needed

node-dist

raw (Proxy)

https://nodejs.org/dist/

For downloading node binaries

node-sass

raw (Proxy)

https://github.com/sass/node-sass/releases/download/

node-sass binaries

node-zopfli

raw (Proxy)

https://node-zopfli.s3.amazonaws.com

node-zopfli binaries

Generally you only need the first two. The other two are dependencies of angular-cli that either need to be build or downloaded as binaries.

We had quite some problems when there was reverse proxy in front of Nexus. Or, actually npm did have problems. So for now we are speaking to nexus3 directly.

Frontend POM

The pom.xml of app-frontend is different from your usual pom.

  1. It doesn’t compile any java or run surefire

  2. It will also clean node/node_modules in clean phase

  3. It will download node binaries and npm installation from nexus

  4. It will run npm install

  5. It will run npm run build to invoke build of the project

  6. It will package the result of the build into a .war file

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <parent>
    <groupId>com.example</groupId>
    <artifactId>parent</artifactId>
    <version>1.0.0-SNAPSHOT</version>
  </parent>

  <artifactId>app-frontend</artifactId>
  <packaging>war</packaging>

  <name>app-frontend</name>

  <properties>
    <!-- (1) this is NOT a java project, therefore we do not compile anything -->
    <maven.main.skip>true</maven.main.skip>
    <maven.test.skip>true</maven.test.skip>
    <!-- this is our Nexus3 installations. No reverse proxy in front -->
    <url.nexus>http://nexus.server:8088/nexus3</url.nexus>
    <!-- proxy of https://nodejs.org/dist -->
    <node.repository>${url.nexus}/repository/node-dist/</node.repository>
    <!-- proxy of NPM registry -->
    <npm.registry>${url.nexus}/repository/npm/</npm.registry>
    <!-- location of npm installation within npm registry -->
    <npm.repository>${npm.registry}npm/-/</npm.repository>
  </properties>

  <dependencies />

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-clean-plugin</artifactId>
        <version>3.0.0</version>
        <configuration>
          <filesets>
            <fileset>
              <!-- (2) during clean phase also clean node_modules of js project -->
              <directory>node/node_modules</directory>
            </fileset>
          </filesets>
        </configuration>
      </plugin>
      <plugin>
        <groupId>com.github.eirslett</groupId>
        <artifactId>frontend-maven-plugin</artifactId>
        <version>1.2</version>
        <configuration>
          <npmVersion>3.10.8</npmVersion>
          <nodeVersion>v6.9.1</nodeVersion>
          <workingDirectory>node</workingDirectory>
          <!-- node executable and npm gets installed under target/ -->
          <installDirectory>target</installDirectory>
          <npmDownloadRoot>${npm.repository}</npmDownloadRoot>
          <nodeDownloadRoot>${node.repository}</nodeDownloadRoot>
          <npmRegistryURL>${npm.registry}</npmRegistryURL>
          <npmInheritsProxyConfigFromMaven>false</npmInheritsProxyConfigFromMaven>
        </configuration>
        <executions>
          <execution>
            <!-- (3) First, install node and npm -->
            <id>install node and npm</id>
            <goals>
              <goal>install-node-and-npm</goal>
            </goals>
            <phase>generate-resources</phase>
          </execution>
          <execution>
            <!-- (4) And then run npm install -->
            <id>npm</id>
            <goals>
              <goal>npm</goal>
            </goals>
            <phase>generate-resources</phase>
            <configuration>
              <!-- The extra arguments specify paths to proxied binaries -->
              <arguments>install --sass-binary-site=${url.nexus}/repository/node-sass/
                       --zopfli_binary_host_mirror=${url.nexus}/repository/node-zopfli</arguments>
            </configuration>
          </execution>
          <execution>
            <!-- (5) And finally do npm run build. We do ng build --aot --prod there -->
            <id>build</id>
            <goals>
              <goal>npm</goal>
            </goals>
            <phase>prepare-package</phase>
            <configuration>
              <arguments>run build</arguments>
            </configuration>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-war-plugin</artifactId>
        <version>2.6</version>
        <configuration>
          <webResources>
            <!-- (6) Put the build results and all the assets into the war -->
            <resource>
              <directory>node/dist</directory>
            </resource>
            <resource>
              <directory>node/assets</directory>
            </resource>
          </webResources>
        </configuration>
      </plugin>
    </plugins>
  </build>

</project>

Tool wrappers

After we have mvn clean install passing and building our frontend, we might want to ensure, that developers will also use the same version of node and npm that our CI does. Since we’re on Windows, we’ll set up two cute wrappers in directory app-frontend/node:

npm.cmd
@echo off
set dir=%~dp0
set node_dir=%dir%\..\target\node
IF NOT EXIST "%node_dir%\node.exe" (
    rem Invoke maven to install the tools
	cd ..
	call mvn frontend:install-node-and-npm
	cd %dir%
)

%node_dir%\node.exe %node_dir%\node_modules\npm\bin\npm-cli.js %*

This ensures that when we invoke npm in project directory the version that Maven will use is used. It also automatically downloads it when not present!

ng.cmd
@echo off
set dir=%~dp0
REM installing angular-cli
set acli_dir=%dir%node_modules\angular-cli\bin
IF NOT EXIST "%acli_dir%\ng" (
   cd ..
   call mvn generate-resources
   cd %dir%
)
set node_dir=%dir%\..\target\node
call %node_dir%\node.exe %acli_dir%\ng %*

We do the same for ng. We cannot just run node_modules\.bin\ng as that would invoke with our system’s node. It is probably much easier to set PATH=..\target\node;node_modules\.bin;%PATH% but you cannot be sure enough with lazy people.

For the less lazy we provide following file they should execute when they start a new terminal for the project:

initenv.cmd
set HTTP_PROXY=
set HTTPS_PROXY=
set PATH=node_modules\.bin;..\target\node;%PATH%
]]>https://pdudits.github.io/2016/11/16/Setting-up-Angular2-CLI-with-Maven-in-enterprise-network.htmlhttps://pdudits.github.io/2016/11/16/Setting-up-Angular2-CLI-with-Maven-in-enterprise-network.htmlWed, 16 Nov 2016 00:00:00 GMT<![CDATA[Java EE 7 Training resources]]>

I’ve done a training with two-day Java EE overview. This is annotated browser history of the things I showed.

At the training we followed book Beginning Java EE 7 from Antonio Goncalves.

The code

We followed code samples from the book troughout the training.

Other general resources

Java EE 7 javadoc lists all of the relevant APIs defined by Java EE spec.

Adam Bien’s book Real World Java EE Patterns – Rethinking Best Practices will help you put EJB and CDI in practice.

Java EE 7 samples demonstrates Java EE 7 features in Arquillian-based tests.

Arquillian helps testing parts of your Java EE applications within container - be it just Weld, embedded Payara or remote standalone servers.

CDI

Read Weld Reference Documentation for in-depth details about all CDI features.

Apache Deltaspike is collection of CDI extensions covering various infrastructure aspects of your application.

JPA

For in-depth look into JPA, design of the entities and trade-offs of the features, read Pro JPA 2 by Mike Keith and Merrick Schincariol.

Another great source of JPA knowledge is Vlad Mihalcea.

Instead of Criteria API, consider using QueryDSL.

For managing of database schema I recommend FlyWay.

Security

Ultimate source on JAAS and JASPIC is Arjan Tijms (and also for JSF and OmniFaces).

Transactions

To understand cases, where distributed transactions are not a good choice, read Your Coffee Shop Doesn’t
Use Two-Phase Commit by Gregor Hohpe.

Also good source for understanding distributed systems, that usually reach outside scope of Java EE, look at Distributed Systems for Fun and Profit by Mikito Takada.

Messaging

Even though Java Serialization is very comfortable, for high performance systems consider other serialization libraries that might give you smaller payloads, less CPU usage, and better interface evolution options.

For building business processes based on message passing you may want to read Enterprise Integration Patterns by Gregor Hohpe (yes, the one of the coffee shop article).

REST

The origin of the term is in Roy Fieldings’ disertation, where it says it’s not just about HTTP, but also about hypermedia.

Serialization frameworks from above may also be a good option for message exchange over REST.

JSF

I think no JSF app can survive without PrimeFaces and OmniFaces. And the book explaining OmniFaces and therefore JSF.

Application Servers

Full listing of certified servers

Standards’ specifications

All Java EE specifications are governed by Java Comunity Process as Java Speficication Requests (JSRs). The list of Java EE 7 JSR with links to spec downloads is available at Oracle’s site

Misc

At heart of every server there is bytecode manipulation library like ASM. Other good use cases are writing tests, that verify correctness of the code (e. g. security check is invoked in every method).

]]>https://pdudits.github.io/2016/08/26/Java-EE-7-Training-resources.htmlhttps://pdudits.github.io/2016/08/26/Java-EE-7-Training-resources.htmlFri, 26 Aug 2016 00:00:00 GMT<![CDATA[400 Bad Request when using Glassfish REST API]]>

When you try to automate your Glassfish administration duties with its REST API using POST or DELETE methods, and all you get is HTTP response 400 and zero content, you forgot to read this:

REST requests that add, update, or delete objects must specify the X-Requested-By header with the value GlassFish REST HTML interface.

It is intended to prevent CSRF attacks as noted in Jason’s Lee post.

]]>https://pdudits.github.io/2012/05/23/400-Bad-Request-when-using-Glassfish-REST-API.htmlhttps://pdudits.github.io/2012/05/23/400-Bad-Request-when-using-Glassfish-REST-API.htmlWed, 23 May 2012 00:00:00 GMT<![CDATA[Enabling SOAP message signing for EJB webservice client in Glassfish]]>

Today’s solution is for following scenario: An EJB uses a web service client, and needs to sign its request with a trusted certificate. We are running Glassfish 3.1.1. Its documentation is pretty straighforward about specifying default client provider, which will cause all webservice calls to be signed. But we cannot do that, because other web service we’re calling cannot handle digitally signed SOAP messages. Documentation only mentions web service endpoint configuration.

Here’s what to do

Add following to your glassfish-ejb-jar.xml:

[source,xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE glassfish-ejb-jar PUBLIC "-//GlassFish.org//DTD GlassFish Application Server 3.1 EJB 3.1//EN"
  "http://glassfish.org/dtds/glassfish-ejb-jar_3_1-1.dtd">
<glassfish-ejb-jar>
    <enterprise-beans>
        <ejb>
            <ejb-name>EjbThatSignsSoapRequests</ejb-name>
            <service-ref>
                <!-- you might need explicit @WebService(name="service") on that field,
                     even if the field is named service -->
                <service-ref-name>service</service-ref-name>
                <port-info>
                    <!-- This is the vital part - specify port of web service -->
                    <wsdl-port>
                        <namespaceURI>urn:webservice:namespace-from-wsdl</namespaceURI>
                        <localpart>WebServicePortName</localpart>
                    </wsdl-port>
                    <message-security-binding
                        auth-layer="SOAP" provider-id="ClientProvider"/>
                </port-info>
            </service-ref>
       </ejb>
   </enterprise-beans>
</glassfish-ejb-jar>

Then, configure your client e. g. via admin gui at path Configurations > server-config > Security > Message Security > SOAP > Tab Providers > Client Provider.

]]>https://pdudits.github.io/2012/04/20/Enabling-SOAP-message-signing-for-EJB-webservice-client-in-Glassfish.htmlhttps://pdudits.github.io/2012/04/20/Enabling-SOAP-message-signing-for-EJB-webservice-client-in-Glassfish.htmlFri, 20 Apr 2012 00:00:00 GMT