Connecting PS4 controllers to PC

I use my 50" TV as my PC screen, which means my peripherals have to be wireless. I recently bought Pummel Party on Steam, and plays better on a controller. My PC doesn't have a Bluetooth card installed, so I went hunting for options. There is this: https://www.playstation.com/en-gb/explore/accessories/dualshock-4-usb-wireless-adaptor/ But what isn't common knowledge is most controllers - ones I have are PS4/XBox/Nintendo Switch - can connect to PC using Bluetooth! I went looking around in shops for the USB wireless adapter, and of course I couldn't find any. Then in a glass display at Qisahn, I find this: https://www.8bitdo.com/wireless-usb-adapter/ Great, it lets me connect all my wireless controllers wirelessly! Brought it home to try, but to my dismay Steam didn't automatically detect my controller as being a PS4 controller. Before upgrading the firmware, it recognized the controller as "Game controller", and after upgrading just became "Controller", but with none of the correct input maps! Not only that but I couldn't map the Analog Stick at all. After more research I got a simple $15 USB-Bluetooth Adapter. Spent a couple of days fumbling to get my controller connected to it. When I tried to connect a second controller, it just wouldn't "stay connected". I was on Windows 7 up until last week and the Bluetooth driver just wasn't working. I even got DS4Windows installed, but made no difference. Since Windows 7 support was ending 14th January, I thought heck now there's probably a legit reason to upgrade to Windows 10 - to get this Bluetooth driver working. I had to uninstall the Bluetooth driver (and unplug it) as well as my Avast Antivirus, otherwise the Windows 10 upgrade would just fail at the BIOS. After upgrade, everything works pretty seamlessly!

Adding ActiveDirectory users to Jenkins

I work(ed) at a Windows-centric organization. Jenkins running on Windows can cause quite a stir. When adding AD users to Jenkins under Configure Global Security, the first problem you’ll encounter is, in Jenkins, users are case-sensitive. That means if your AD user is JOHN, you’ll need to add both JOHN and john, otherwise when the user decides to login with small case, it won’t work. A bigger problem is, once you cross about 50 users, you’ll start getting exception as documented in JENKINS-26963 - Form too large 213549>200000. The quick fix is to add a JVM parameter to jenkins.xml. If you’re running Jenkins behind jetty on Windows, you actually need to do this instead: prunmgr.exe //ES//Jenkins (You can get the edit string from Windows services) Funnily enough, if you visit https://wiki.jenkins.io/display/JENKINS/Jetty, the lone comment on that page addresses the above. Considering Jenkins is bundled on Jetty you would think this was better documented.

Mono-repo with Lerna

Dev team wanted to use lerna - defined bootstrap and postinstall commands that called lerna. Lerna bootstrap default behaviour uses npm ci This will fail if no package-lock.json. If you set postinstall this will never exist Package-lock not sustainable in git when many developers contributing code Thus will fail with nipm package-lock doesnt exist. We had to use —no-ci and sacrifice the speed boost. Lerna bootstrap will also run forever if downstream npm install demands output. For eg in my case semantic.json had backslashes for paths, and gulp returns a prompt - semantic.json exists do you want to Skip Install. Lerna with loglevel silly will get stuck at ‘npm install’ on the leaf and not say anything. I found advise about not putting lerna bootstrap in a postinstall command, however that is no longer applicable. Lerna publish creates git tags for every subpackage but only if its changed, so you end up with a mess of tags with different versions. So to make sure we have 1 version for everything in the repository, I use sed to replace version in the root and packages package.json, but also change the versions of local dependencies. Lerna bootstrap calls node-gyp rebuild, which must connect to internet to download node. Unless you set nodeconfig to a local installation. Easiest way was to set config in .npmrc. Since lerna uses webpack, I got away with installing this globally. For gulp, even if I installed globally, it still complained the command didn’t exist. The solution was to —save-dev and make it a devDependency. Some places actually suggest to not use lerna bootstrap and switch to using file specifiers for local dependencies. This ends up a greater headache - lerna bootstrap downloads a ton of gulp dependencies (gulp-help, gulp-concat, etc.). Don’t heed that advice.

Generating SHA-256 checksums for Maven artifacts

This one is thoroughly undocumented. Didn't go through the plugins code to work this one out, it worked purely by chance...

My organization requires that for all artifacts to be released, a SHA-25 checksum needs to be generated. I've standardized on pom.xml for all projects in order to upload artifacts to Nexus. My alternative was to upload artifacts via Jenkins in a pipeline using the Nexus uploader block, however it doesn't seem there's a simple way to get identify ahead of time the artifacts that would be built in the Maven dependency tree. If there was, I could just run "sha256sum" on this list... I did try suggestion from here but if I recall it didn't list artifacts from child modules: https://stackoverflow.com/questions/36936238/create-a-list-of-artifacts-that-are-build-by-a-maven-project

Far easier to use a Maven plugin with "mvn deploy". Cue the checksum-maven-plugin:
https://checksum-maven-plugin.nicoulaj.net/examples/generating-project-artifacts-checksums.html

Looks simple enough. Quote: "This configuration will generate checksum digest files for the project main and attached artifacts".

      <plugin>
        <groupId>net.nicoulaj.maven.plugins</groupId>
        <artifactId>checksum-maven-plugin</artifactId>
        <version>1.8</version>
        <executions>
          <execution>
            <goals>
              <goal>artifacts</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <!-- put your configurations here -->
        </configuration>
      </plugin>

When I tried this plugin, no SHA-256 checksum gets generated.

Within the responses on the Github repository for that plugin, I find this:
https://github.com/nicoulaj/checksum-maven-plugin/issues/39

<plugin>
    <groupId>net.ju-n.maven.plugins</groupId>
    <artifactId>checksum-maven-plugin</artifactId>
    <version>1.3</version>
    <executions>                   
        <execution>
            <id>checksum-artifacts</id>
            <phase>package</phase>
            <goals>
                <goal>artifacts</goal>
            </goals>
            <configuration>
                <csvSummary>false</csvSummary>
                <shasumSummary>true</shasumSummary>
                <shasumSummaryFile>sha512-libs.sum/shasumSummaryFile>
                <individualFiles>false</individualFiles>
                <algorithms>
                    <algorithm>SHA-512</algorithm>
                </algorithms>
                <types>
                    <type>jar</type>
                </types>
                <scopes>
                    <scope>runtime</scope>
                </scopes>
            </configuration>
        </execution>
    </executions>
</plugin>

I guess it used to have a different group name prior to version 1.5, however this version of the plugin was at least printing a line in the Maven output indicating this plugin was getting invoked. I thought my plugin wasn't even getting called! However still no checksum was getting generated.

I'd almost given up, until my colleague started using the plugin and SHA-256s were getting generated and auto-uploaded to Nexus. After a bit of digging, I found that only artifacts in the "${workspace}/target" directory were getting checksums generated. My artifacts were getting generated in child module folders, and any arbitrary directory the project called for - e.g. from using maven exec plugin or antrun.

The solution was to add an extra antrun step to move any built artifact into the project root's target directory, and then use attach-artifacts to include it for upload to Nexus. In some of my cases, I had to add an extra module to perform this after other child module's had completed building. Sometimes the Maven reactor wouldn't order the child module's properly, especially if the module's were built with proprietary Maven plugins (e.g. Temenos products). Not the user-friendly experience I expected for generating checksums!

MySQL driver and Fuse

The internet caused me a headache this past week.
All the guides on using MySQL with Fuse in a project utilizing Blueprint DSL will demonstrate something like this:
1. In your POM declare <Import-Package>com.mysql.jdbc</Import-Package>
2. Install mysql-connector to your OSGi container using "osgi:install"
3. Install your app.

Here's some links: Fuse examples on Git, http://stackoverflow.com/questions/30307288/mysql-connector-in-osgi-environment-gradle-noclassdeffounderror, http://freemanfang.blogspot.sg/2012/03/how-to-use-jdbc-driver-in-osgi.html, http://www.liquid-reality.de/display/liquid/2012/01/13/Apache+Karaf+Tutorial+Part+6+-+Database+Access

If you were getting error "java.lang.ClassNotFoundException: com.mysql.jdbc.Driver not found", you would probably come across the above guides. However, you would be severely misled, because they all do not address the fundamental issue: MySQL changed their package name of the Driver.class.
I guess this is one of the downfalls of proprietary libraries, they change package names at will, and there are 0 OSGi articles that mention this. So the easy fix to your solution would be:
1. Change this in your POM:

      <plugin>
        <groupId>org.apache.felix</groupId>
        <artifactId>maven-bundle-plugin</artifactId>
        <version>${version.maven-bundle-plugin}</version>
        <extensions>true</extensions>
        <configuration>
          <instructions>
             <Bundle-SymbolicName>${project.artifactId}</Bundle-SymbolicName>
             <Bundle-Description>${project.description}</Bundle-Description>
             <Import-Package>com.mysql.cj.jdbc, com.ibm.mq.jms, com.ibm.mq, com.ibm.mq.constants ,org.springframework.jdbc.*, org.apache.commons.dbcp,*;resolution:=optional</Import-Package>
             <DynamicImport-Package>*</DynamicImport-Package>
          </instructions>
        </configuration>
      </plugin>

2. Change this in your blueprint:

    <bean class="org.apache.commons.dbcp.BasicDataSource" id="dataSource">
        <property name="driverClassName" value="com.mysql.cj.jdbc.Driver"/>
        <property name="url" value="jdbc:mysql://localhost:3306/yourdb"/>
        <property name="username" value="user"/>
        <property name="password" value="pass"/>
    </bean>

I really hope this saves some people a heap of time!!

SoapUI and MQ on Windows

WebSphere MQ is a fairly complex piece of software, with concepts ranging from Connection Factories, Topics, Subscribers, Channels, etc. So getting SoapUI to connect to it requires a decent amount of technical know-how.
There's ample SoapUI documentation on picking up & sending messages to ActiveMQ, however for IBM MQ it's a bit sparse. This is an attempt to document, step by step, how to get SoapUI to quickly hook up to MQ, in Windows 7, taking into consideration UAC.

1. Get WebSphere MQ Developer edition, if you don't have it already. I made the mistake of getting the evaluation trial.
2. If working locally, create a queue manager in MQ, then open up command prompt, run "runmqsc ", and type "ALTER QMGR CHLAUTH(DISABLED)". You don't need to worry about channel authentication for development work, but if you insist, it took me some time to figure this out but you need to first create a server-connection channel (you had the option to do this on installation of MQ), open channel properties and under MCA, replace *NOACCESS with MUSR_MQADMIN (if using default domain/users).
3. Go to %SOAPUI_HOME%/bin, open up "soapui.bat", and edit this line so it becomes:
   set CLASSPATH=%SOAPUI_HOME%soapui-5.3.0.jar;%SOAPUI_HOME%..\lib\*;C:\Program Files\IBM\WebSphere MQ\java\lib\*
   Then run this bat file as Administrator.
4. Load up a WSDL in SoapUI. To save yourself some time, use the sample SoapUI SOAP tutorial which comes with the SoapUI installation. On Windows this is put in C:/Users/username/SoapUI-Tutorials by default.
5. Run HermesJMS from within SoapUI. Configure the path to HermesJMS when it prompts you.
6. Create a new session. This guide can take you through *most* of the way: Guide
   But you might get numerous errors about classes not being runnable. In my classpath group I ended up with this, just to be sure.
   
   
7. If you left channel authentication on, you need to connect through the server-connection channel. Your session configuration needs to look like this:
   
   
   
   (And yes that's a Mac UI, Mac users can follow these instructions)
8. Be careful not to leave "MQQueueConnectionFactory" as the Connection Factory for the session if you're getting classpath errors, otherwise your session will become corrupt and you'll need to delete the HermesJMS hermes-config.xml file and start over.
9. Right click session, Discover, and HermesJMS should find all your queues.

Finding the Eclipse test client URL

Generate a bottom-up web service and deploy to Eclipse Tomcat, and the internal browser automatically pops up:

Now, what if you close that browser? Well you're in a predicament, you either have to:
- regenerate the test client project
- guess the URL of the sample test project

This caused me much grief, so for reference, here's the URL:
http://localhost:/Client/sampleProxy/TestClient.jsp
Where your port is either the Tomcat port or the monitor port.

Oracle Linux & yum

Recently my wife had brought back a laptop with an Oracle Linux VM running on VMWare Fusion. It was based off RHEL 4.4 which I thought was pretty darned old. She was undergoing training at her new job and the class were trying to install the "screen" command with yum. Apparently not even the trainer could figure out how to get "yum" to work.

The first thing I noticed was the Red Hat subscription message:
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
Setting up Install Process
No package screen available.

I'd never setup an RHEL OS before, so I looked for online repos to add to yum.repos.d, and tried to enable the "rhel-source" repo that was available by default, but no dice.

After a bit of playing around, I realized I had to activate Red Hat subscription. I was quite surprised the company would give out VMs with expired/no subscriptions to freshies, for training! Instructions here: https://access.redhat.com/solutions/253273

First you have to create an account on the Red Hat portal. You can choose "Personal" or "Corporate", this confused me for awhile because you can't get an Evaluation Subscription without a corporate email address. This was restrictive based on the email address you provided (I gave a gmail account), but then changed it to her company's provided one. Then you have to activate the subscription, otherwise you'll get: <user> cannot register with any organizations. Once you have a subscription, you'll be able to run "yum install screen" (or any package for that matter!)

Hope this helps someone!

Java 7 and SSL

We faced this at work a few days ago.
We run a Java standalone application, with embedded JBoss. It runs on Java 7. One of our customers insisted on TLSv1.2 protocol for their server.
Now, the issue with this is Java 8 has TLSv1.2 enabled by default. Java 7 doesn't, it's only got SSLv2 and 3 enabled by default.
Typically, just setting something like "-Ddeployment.security.TLSv1.2=true" to run your application would be sufficient, however our coders actually hardcoded the SSL context so this never worked...

Docker Machine on Windows

After having so much fun with running Docker in a Virtualbox VM, I decided to explore Docker's solution to running Docker on Windows. Turns out, there's really not much different, apart from using 'docker-machine create' to link Windows Docker commands to the VM one.

I run the Docker Quickstart terminal, which creates a 'default' VM under IP 192.168.99.100, with Docker installed and daemon running on port 2376. It has a NAT Network Adapter, with a randomly forwarded port (e.g. 56858) to guest port 22. It also creates a Host-Only Network Adapter in Virtualbox, named "Virtualbox Host-Only Ethernet Adapter", and this has a DHCP server enabled to distribute IPs. The adapter has an IP address of "192.168.99.1". The rest looks like this:

I had to figure all this out myself, and I really wish all this information was just laid out from the start on the Docker documentation.

My goal was to create a Virtualbox VM, and hook up Windows Docker commands to it. This gives me the flexibility of starting a VM via Vagrant, and not having to use boot2docker OS. Seems logical to me.

First, this line is required in Vagrantfile:

config.vm.network "private_network", type: "dhcp"

This will create a Host-Only network for you, meaning the VM will have an IP assigned by Virtualbox's DHCP server. I don't know about you, but I get an automatically assigned IP: 172.28.128.1

Then you need to somehow automatically determine this IP. You can use this:

"vagrant ssh -c "ip address show eth1 | grep 'inet ' | sed -e 's/^.inet //' -e 's/^([0-9.]+)./\1/'"

Note how I assume 'eth1'. This is because I expect to only have 2 adapters, and eth0 is used by the NAT adapter.

Knowing the IP of your VM, you can run this on Windows (you must pass in the private key to generic-ssh-key):

docker-machine -D create --driver generic --generic-ssh-user root --generic-ssh-key myfolder/id_rsa --generic-ip-address 172.28.128.4 --generic-ssh-port 22 myserver

But then you may encounter this:

Error running SSH command: exit status 127

This actually requires you to put your public key into the 'authorized_keys' file for that user (in my case, root) on your VM.

After that, you may get this:

Reading server key from C:\Users\Alkaiser\.docker\machine\machhefserver\server-key.pem
Error creating machine: Error checking the host: Error checkinor regenerating the certs: There was an error validating certis for host "172.28.128.4:2376": dial tcp 172.28.128.4:2376: i/out
You can attempt to regenerate them using 'docker-machine regencerts [name]'.
Be advised that this will trigger a Docker daemon restart whic stop running containers.

Other users suggest you have a conflicting Host-Only adapter. I wouldn't rule this out, however it is more likely your TCP connection is being blocked. You can validate this by running "telnet 172.28.128.4 2376". This should connect because the Docker daemon is listening on that port. If this doesn't work, it means your VM is blocking that port. On CentOS7, I unblock it by using:

firewall-cmd --permanent --zone=public --add-port=2376/tcp; systemctl restart firewalld

Now you should get (with debug output):

Docker is up and running!
Reticulating splines...
(chefserver) Calling .GetConfigRaw
To see how to connect your Docker Client to the Docker Engine g on this virtual machine, run: D:\Program Files\Docker Toolboer-machine.exe env chefserver
Making call to close driver server
(chefserver) Calling .Close
Successfully made call to close driver server
Making call to close connection to plugin binary
Making call to close driver server
(flag-lookup) Calling .Close
Successfully made call to close driver server
Making call to close connection to plugin binary

Have fun with your custom Docker machine!

Setting up Chef with Vagrant box

I certainly ran into some frustrating scenarios with this setup, either because my Engrish isn't good, or documentation isn't as comprehensible as it should be (for Chef).
Here's what I wanted to achieve:
- Vagrantfile to spin up a Virtualbox instance
- Chef Server on Virtualbox instance
- Chef Client on Windows workstation

This was my guide, pretty good one at that: https://www.digitalocean.com/community/tutorials/how-to-set-up-a-chef-12-configuration-management-system-on-ubuntu-14-04-servers
1. I create a simple Vagrantfile with 'vagrant init'. Easy.
Also I note the default Vagrantfile has some stuff about setting up Chef Solo and linking to an existing Chef Server. However it has nothing along the lines of creating my own Chef Server in the Vbox instance. Shame...
2. 'vagrant up' and follow steps on digitalocean website. I skipped adding a hostname initially since vagrant instances already add the hostname to 127.0.0.1, but later added it in during troubleshooting. Shouldn't make any difference unless SSH-ing from a machine not on localhost.
3. Install Chef Server using rpm, then ran chef-server-ctl reconfigure, takes 10-15 minutes but all seemed functional.

It comes around to Chinese New Year and some spring cleaning was in order, so my PC gets shut off. I now start up my VBox instance again, but I wonder how to start the chef-server, if that's even necessary.
So I do some googling and find it's started with 'chef-server-ctl reconfigure', and I run this. However after a few hours, NOTHING HAPPENS. Nothing in /etc/init.d either. Turns out it's using an embedded nginx server...surely it's not this hard?
After a night's sleep, I figure out why. It's because vagrant started my virtualbox instance with 633MB of memory. Bugger that! I also missed this part: The Chef documentation tells us that your Chef server should have at least 4 cores and 4 GB of RAM; That's quite a bit...anyway I bumped it up to 2GB and reconfigure takes a couple of minutes, yay!

4. Create a simple chef-repo, put it in git. Setup Chef DK on my Windows PC. So far so good.
5. Create a couple of .pem files on server as per guide. I just copy them over to workstation via /vagrant folder, no biggie.
6. Create knife.rb in my .chef folder, and run knife client list. But instead of "certificate verify failed" like the guide says, I get "unknown protocol". Continuing to the next step, "knife ssl fetch", that yields the same result. Ah shit, I've now skipped some of the previous steps like setting up SSH keys, where have I gone wrong!
7. I setup all my SSH keys, setup hosts in /etc/hosts like a studious boy, and make sure normal SSH works like 'vagrant ssh' would...oh right default vagrant SSH port is '2222', maybe that could be it...
But if you look at the knife.rb file, it looks like this:

current_dir = File.dirname(__FILE__)
log_level                :info
log_location             STDOUT
node_name                "admin"
client_key               "#{current_dir}/admin.pem"
validation_client_name   "digitalocean-validator"
validation_key           "#{current_dir}/digitalocean-validator.pem"
chef_server_url          "https://server_domain_or_IP/organizations/digitalocean"
syntax_check_cache_path  "#{ENV['HOME']}/.chef/syntaxcache"
cookbook_path            ["#{current_dir}/../cookbooks"]

Hmmm, but it's using "https" so it must use port 443, and it's not using SSH...ok so nginx is exposing a chef webservice on port 443.
Maybe my certs were generated with the wrong host in /etc/hosts file...I regenerate the .pem files and put them in my .chef folder...no that ain't working either!!
Oh. I haven't port forwarded port 443 from Virtualbox. So I set my host port to 4443 and guest to 443. Voila that let me use 'knife ssl fetch'! 'knife client list' is showing me what I want too.
8. I run this command:
knife bootstrap vagrant-centos65.vagrantup.com:2222 -N testing -x vagrant -P vagrant --sudo --use-sudo-password
It starts up Chef Client version 12.7.0 and does this:

vagrant-centos65.vagrantup.com      [2016-02-11T17:37:09+00:00] ERROR: Error connecting to https://vagrant-centos65.vagrantup.com:4443/organizations/myapp/nodes/testing, retry 1/5
vagrant-centos65.vagrantup.com      [2016-02-11T17:38:17+00:00] ERROR: Error connecting to https://vagrant-centos65.vagrantup.com:4443/organizations/myapp/nodes/testing, retry 2/5
vagrant-centos65.vagrantup.com      [2016-02-11T17:39:25+00:00] ERROR: Error connecting to https://vagrant-centos65.vagrantup.com:4443/organizations/myapp/nodes/testing, retry 3/5
vagrant-centos65.vagrantup.com      [2016-02-11T17:40:33+00:00] ERROR: Error connecting to https://vagrant-centos65.vagrantup.com:4443/organizations/myapp/nodes/testing, retry 4/5
vagrant-centos65.vagrantup.com      [2016-02-11T17:41:41+00:00] ERROR: Error connecting to https://vagrant-centos65.vagrantup.com:4443/organizations/myapp/nodes/testing, retry 5/5
vagrant-centos65.vagrantup.com
vagrant-centos65.vagrantup.com      ================================================================================
vagrant-centos65.vagrantup.com      Chef encountered an error attempting to load the node data for "testing"
vagrant-centos65.vagrantup.com      ================================================================================
vagrant-centos65.vagrantup.com
vagrant-centos65.vagrantup.com      Networking Error:
vagrant-centos65.vagrantup.com      -----------------
vagrant-centos65.vagrantup.com      Error connecting to https://vagrant-centos65.vagrantup.com:4443/organizations/myapp/nodes/testing - Connection timed out - connect(2) for "vagrant-centos65.vagrantup.com" port 4443
vagrant-centos65.vagrantup.com
vagrant-centos65.vagrantup.com      Your chef_server_url may be misconfigured, or the network could be down.

Oh noooo what's happening...what's worse is if I "Ctrl+C" out of the timeout I get this:
vagrant-centos65.vagrantup.com [2016-02-11T17:29:17+00:00] WARN: Chef client 18958 is running, will wait for it to finish and then run.
Firstly, what is this PID? "ps -ef" on Cygwin, and Task Manager doesn't show any PID anywhere near that number.
I find this blog and start searching for a "chef-client-running.pid" file on my system. The code references on the blog look a little outdated so I search the code for references to that file, and I find this:

/cygdrive/c/opscode
$ grep -ir "chef-client-running.pid"
...
chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/spec/unit/run_lock_spec.rb:  default_pid_location = windows? ? 'C:\chef\cache\chef-client-running.pid' : '/var/chef/cache/chef-client-running.pid'

Sure enough, there's a PID file in that location. I delete the file and re-run knife.
Knife didn't seem to care about that file, it still presented itself with the "will wait for it to finish" message. After a day of hunting around on my system, I take a pause and realize 'knife' is actually doing an SSH to my Virtualbox instance, and running chef-client there! I run 'vagrant ssh' and a 'ps -ef | grep chef' on the server and yes, there's that dang PID!
Alright so now I can break out of a hanging knife. But why's it hanging? Turns out when I use port "443" to access this URL "https://vagrant-centos65.vagrantup.com/organizations/myapp" it all works. I'm guessing the certificates that were generated force us to use port 443 on the host machine ... either that or knife bootstrap really wants to use port 443. Anyway that resolved the problem and now I get:

~/myapp/gitrepo/chef-repo
$ knife bootstrap -V vagrant@vagrant-centos65.vagrantup.com:2222 -N testing -x vagrant -P vagrant --sudo --use-sudo-password
INFO: Using configuration from D:/cygwin64/home/Alkaiser/myapp/gitrepo/chef-repo/.chef/knife.rb
Doing old-style registration with the validation key at D:/cygwin64/home/Alkaiser/myapp/gitrepo/chef-repo/.chef/myapp-validator.pem...
Delete your validation key in order to use your user credentials instead

Connecting to vagrant-centos65.vagrantup.com:2222
vagrant-centos65.vagrantup.com      -----> Existing Chef installation detected
vagrant-centos65.vagrantup.com      Starting the first Chef Client run...
vagrant-centos65.vagrantup.com      Starting Chef Client, version 12.7.0
vagrant-centos65.vagrantup.com
vagrant-centos65.vagrantup.com      ================================================================================
vagrant-centos65.vagrantup.com      Chef encountered an error attempting to load the node data for "testing"
vagrant-centos65.vagrantup.com      ================================================================================
vagrant-centos65.vagrantup.com
vagrant-centos65.vagrantup.com      Authentication Error:
vagrant-centos65.vagrantup.com      ---------------------
vagrant-centos65.vagrantup.com      Failed to authenticate to the chef server (http 401).
vagrant-centos65.vagrantup.com
vagrant-centos65.vagrantup.com      Server Response:
vagrant-centos65.vagrantup.com      ----------------
vagrant-centos65.vagrantup.com      Failed to authenticate as 'testing'. Ensure that your node_name and client key are correct.
vagrant-centos65.vagrantup.com
vagrant-centos65.vagrantup.com      Relevant Config Settings:
vagrant-centos65.vagrantup.com      -------------------------
vagrant-centos65.vagrantup.com      chef_server_url   "https://vagrant-centos65.vagrantup.com/organizations/myapp"
vagrant-centos65.vagrantup.com      node_name         "testing"
vagrant-centos65.vagrantup.com      client_key        "/etc/chef/client.pem"
vagrant-centos65.vagrantup.com
vagrant-centos65.vagrantup.com      If these settings are correct, your client_key may be invalid, or
vagrant-centos65.vagrantup.com      you may have a chef user with the same client name as this node.
vagrant-centos65.vagrantup.com
vagrant-centos65.vagrantup.com
vagrant-centos65.vagrantup.com      Running handlers:
vagrant-centos65.vagrantup.com      [2016-02-13T05:22:44+00:00] ERROR: Running exception handlers
vagrant-centos65.vagrantup.com      Running handlers complete
vagrant-centos65.vagrantup.com      [2016-02-13T05:22:44+00:00] ERROR: Exception handlers complete
vagrant-centos65.vagrantup.com      Chef Client failed. 0 resources updated in 07 seconds
vagrant-centos65.vagrantup.com      [2016-02-13T05:22:44+00:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
vagrant-centos65.vagrantup.com      [2016-02-13T05:22:44+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
vagrant-centos65.vagrantup.com      [2016-02-13T05:22:44+00:00] ERROR: 401 "Unauthorized"
vagrant-centos65.vagrantup.com      [2016-02-13T05:22:44+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

Unauthorized? Oh I registered the node and removed the node from my client so I could reproduce this for my blog...ehem. This kind soul here helped me figure it out. Delete /etc/chef/client.pem from your server and re-run knife bootstrap. Now I get "Chef Client finished, 0/0 resources updated in 09 seconds", alright!

Now I want to run some playbooks. More frustration...
8. Run "knife cookbook upload -a". This actually required me to manually clone the different dependant projects (e.g. Tomcat relies on Java which relies on yum-epel and openssl etc.) before this would even work. Surely there's an automated way to do this...?
UPDATE: Just so I don't confuse anyone, yes you can do this with "knife cookbook site install COOKBOOK_NAME [COOKBOOK_VERSION] (options)"
9. Run "knife node edit testing" to update the run list, so I can actually install stuff. Instead I get this
ERROR: RuntimeError: Please set EDITOR environment variable
So I find out I have to set knife[:editor] in my knife.rb file. I set it to the long filepath to Notepad++.exe, however I just keep getting the same error, or this:
syntax error, unexpected tSTRING_BEG, expecting end-of-input
Argh...so in the end I finally found this:
https://tickets.opscode.com/browse/CHEF-4503
Looks like you MUST set the value to a Windows shortname using 8.3 notation. It ended up looking like this (the options are mandatory for this to work):
knife[:editor] = "D:\\PROGRA~1\\NOTEPA~1\\NOTEPA~1.EXE -nosession -multiInst"

Where on earth is this in the documentation for that here https://docs.chef.io/config_rb_knife.html??? For the love of ...
10. Run "chef-client" as root on the target server. Hm can't I manage the node remotely? Yes you can, I ended up with this:
knife ssh "name:testing" "sudo chef-client" -x vagrant -p 2222
In the end you get this:

Running handlers:
Running handlers complete
Chef Client finished, 14/15 resources updated in 04 minutes 51 seconds

Yay!
Extra point: I had to add the same version of the Guest Additions ISO that I had on my base box to my Vagrantfile. For 4.3.14 for example, I downloaded it from here: http://download.virtualbox.org/virtualbox/4.3.14/VBoxGuestAdditions_4.3.14.iso, and put in the Vagrantfile this config: config.vbguest.iso_path=

Corsair Void Headset

Short post in case anyone has the same product.
Bought a Corsair VOID RGB headset at $180, looks and sounds great! http://www.corsair.com/en-us/landing/void
I'd be playing music and suddenly I'd get my microphone beeping red rapidly, and a few seconds later it'd just shut off. I'd turn it on again and the side lights wouldn't go on...
I saw a setting on the CUE control panel called "Disable auto shutoff" disabled, which I find really weird for a default setting. Anyway I've now enabled it, hopefully I don't face the issue anymore!

Setting up webserver on RasPi

This shouldn't have been a headache, but it sure was!
The idea was:
1. Get a domain
2. Put my site content on RasPi
3. Profit
I'd recently moved to Singapore from Australia, so naturally a new router. I'm actually surprised by the amount of changes I needed to move from my previous DLink router to the new DLink router. Anyway this is what I ended up having:

auto lo

iface lo inet loopback
iface eth0 inet dhcp

wireless-power off
allow-hotplug wlan0
auto wlan0

iface wlan0 inet manual
   wireless-essid 'MyInternetz'
   wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf

iface home inet static
   address 192.168.0.250
   netmask 255.255.255.0
   network 192.168.0.0
   gateway 192.168.0.1

iface default inet dhcp

I'd actually struggled with stability with the RasPi. The keywords are "wireless-power off". The /var/logs/network said nothing about power saving...
So I go on to buy a domain off NameCheap, and learn a thing or two about CNAMEs. Hence I create one that points to my IP...oh hang on, I need a static IP which typically means I need to pay my ISP a few more dollars...lets not.

I set up NoIP and use their dynamic IP updater so I can set a hostname which points to my dynamic IP. This is where I then realize my ISP has some paranoia issues and blocked port 80! No problem, I'll setup a URL Redirect record in NoIP, then on NameCheap my CNAME can point to this URL Redirect. If only it were that simple, it doesn't look like CNAMEs can point to URL Redirects, not quite sure why this doesn't work...

Luckily, NameCheap have their own dynamic IP updater which I hadn't realized earlier, so I scrap the NoIP idea. This creates an A+ dynamic DNS record which points to my dynamically updated IP. Unfortunately, this still means I can't use port 80, because CNAMEs don't accept ports, and neiher does the dynamic IP updater...I've decided to live with specifying ports for now.

What I find wrong with MacOS

Just a rant. Since using a Mac I question why all the hype about using Macs over Windows. There are interface problems that persist that no one is reviewing and irritate the hell out of me. And then some problems are probably just personal preference...

• On Date & Time preferences, why do I need to "unlock to make changes", and then untick "Set date and time automatically" (which lags by the way), just to view other months on my calendar?
• If for some reason a script on my browser takes awhile to run, my cursor clicks go mental. E.g. if I click on a file to rename it, Mac immediately unfocuses my highlight, so quickly that I can't rename my file. Or if I highlight a file, I lose my highlight almost immediately. There must be something wrong with the multi-threading...
• When I minimize a window, why does a new window get created in my dock?
• Can't resize images in Outlook. Microsoft hurry up with the update.
• The closest thing to a Notepad++ (free text editor) I could find for Mac is Brackets, which is lacking alot of features.
• Fn+F11 is NOT how I want to shimmy to Desktop (given minimizing is not an option either), give me a handy button in the Dock.
• Opening programs in the Dock require only a single-click to open. So I get alot of misfires and get frustrated.
• Format Painter in Microsoft products on Mac are on the top, but on Windows it's in the ribbon. This is more a Microsoft inconsistency issue...
• If I drag a file into a folder on Finder, please put it into that folder, NOT the folder I am currently in. Jeez!
• Scrolling horizontally requires 2 fingers to drag horizontally, which is great, except on a browser this pushes me back a page as well.
• Volume keys randomly stop working. I get the 'restricted' sign instead. "sudo killall coreaudiod" fixes this. What on earth?

Splunk & Spring Integration

Today's headache is integrating the two!

Me, having limited Spring experience, embarked on a wild journey of bean definitions and namespace resolutions, it got really frustrating before enjoyable...

Referencing http://docs.spring.io/autorepo/docs/spring-integration-splunk/0.5.x-SNAPSHOT/reference/htmlsingle/, it seems easy. Add this and you're set, or are you?

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xmlns:int="http://www.springframework.org/schema/integration"
 xmlns:int-splunk="http://www.springframework.org/schema/integration/splunk"
 xsi:schemaLocation="http://www.springframework.org/schema/integration/splunk
  http://www.springframework.org/schema/integration/splunk/spring-integration-splunk.xsd
  http://www.springframework.org/schema/integration
  http://www.springframework.org/schema/integration/spring-integration.xsd
  http://www.springframework.org/schema/beans
  http://www.springframework.org/schema/beans/spring-beans.xsd">

</beans>

Well no, because going to http://www.springframework.org/schema/integration/splunk yields nothing. I'd wish Spring mentions that!

You must add this to your dependencies:
http://maven-repository.com/artifact/org.springframework.integration/spring-integration-splunk/1.1.0.RELEASE
Cool so for me, I'm using IntelliJ, and Ivy for my dependency management. Keep in mind IntelliJ Community Edition has no Spring support, thought I'd mention that because my Intellij was complaining about Spring being an "unknown facet". Also, I'm running my application on Jetty.

It should just work, but I continually got this:
Caused by: org.springframework.beans.FatalBeanException: Class [org.springframework.integration.splunk.config.xml.SplunkNamespaceHandler] for namespace [http://www.springframework.org/schema/integration/splunk] does not implement the [org.springframework.beans.factory.xml.NamespaceHandler] interface
What does this even mean? After lots of googling, I was led to believe I had a classloader issue. Do I add an entry to my web.xml, or my project classpath? Do I add my JAR to my WEB-INF/lib folder (worst suggestion ever)? But everything in my IntelliJ lib folder is already on the classpath!! Then I thought, hm maybe the error means something and it's because SplunkNamespaceHandler is extending AbstractIntegrationNamespaceHandler...

Nope, none of the above. The problem is the Splunk JAR (1.1.0) has a dependency on 4.0.2-RELEASE of Spring, whereas my Spring context.xml looked like the below:

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:context="http://www.springframework.org/schema/context"
       xmlns:task="http://www.springframework.org/schema/task"
       xmlns:util="http://www.springframework.org/schema/util"
       xmlns:int-splunk="http://www.springframework.org/schema/integration/splunk"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
       http://www.springframework.org/schema/beans/spring-beans-3.1.xsd
       http://www.springframework.org/schema/integration/splunk
       http://www.springframework.org/schema/integration/splunk/spring-integration-splunk.xsd
       http://www.springframework.org/schema/context
       http://www.springframework.org/schema/context/spring-context-3.1.xsd
       http://www.springframework.org/schema/task
       http://www.springframework.org/schema/task/spring-task-3.0.xsd http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd">

Changing all the Ivy dependencies to 4.0.2 and making sure IntelliJ's default Spring libraries aren't being used did the trick.

Also, my perfect girlfriend asked for a shoutout, so here it is! :)

Packer, Vagrant and Windows

Fun day of setting up Packer with configurations my colleague put together :)
Packer is a way for you to build a Vagrant box locally with all the software and configurations you need, without having to transfer an enormous VBox around. Very neato.
On running a simple "packer build ", your OS should build from scratch. First problem I encounter:
More info here
The executable 'bsdtar' Vagrant is trying to run was not
found in the %PATH% variable. This is an error. Please verify
this software is installed and on the path.

What do you expect I'd do? I locate bsdtar.exe in "C:\HashiCorp\Vagrant\embedded\mingw\bin", and add the path to my PATH. Then I get another error:

The box failed to unpackage properly. Please verify that the box
file you're trying to add is not corrupted and try again. The
output from attempting to unpackage (if any):

x Vagrantfile
x box.ovf
x metadata.json
x ubuntu1404-disk1.vmdk: Write failed
Packer/bsdtar.EXE: Error exit delayed from previous errors.

Well that was useless. Long story short, it seems like a bug when upgrading Vagrant from an older version to a newer version. I upgraded from 1.6.3 to 1.6.5. Uninstalled my current and reinstalling Vagrant 1.6.5 fixed the issue.

Getting past the intial setup, we wrote scripts to automate installation of a particular IBM product: Maximo. Here's some of the dependencies:

• Install WebSphere Application Server or Oracle WebLogic, ~2-3GB
• A database (I used Oracle 11g R2), minimalistically this takes about 5-10GB space
• Yum packages (e.g. Ant, Oracle DB pre-reqs)
• Open File descriptors, kernel property changes
• Running maxinst.sh

Installing Weblogic and Oracle DB doesn't actually taking a long time. Maxinst takes the bulk of the time, and has a tendency to fail. A couple of key notes I took for Packer:

• The documentation suggests using a post processor to keep "intermediary artfacts" (the vbox) like so:
  

  
    "post-processors": [
      {
        "output": "builds/centos65-wwm-base.box",
        "type": "vagrant",
        "keep_input_artifact": true
      }
    ]
  

  
  The trouble is, I still get "Deleting output directory" at the end of a failed build, which means "keep_input_artifact" only works if your build succeeds (I'm guessing, I never tried). Horrible stuff, you're going to automatically delete 3 hours worth of builds with no way for me to keep my vbox? Not happy HashiCorp.
• I like to lock my screen while stuff runs in the background. With Packer? Bad idea.
  
  

SoapUI working with IBM JRE

In short: there is no support from Smartbear to support the IBM JRE, all efforts lead to a response of "use the Sun JRE".
Why would you use the IBM JRE? This is to send JMS messages to WebSphere's SI Bus, where the Application Server has Global Security turned on. You are required to set these 2 JVM properties:
-Dcom.ibm.CORBA.ConfigURL
-Dcom.ibm.SSL.ConfigURL

If you don't do this and attempt to send a message, you get a WsnInitialContext exception.
Once you've configured soapui-pro.sh to use the IBM JRE, you'll find that you won't be able to activate/use your license (even if you'd activated it while using the Sun JRE). You'll go through the process of re-activating your license, but be told you're missing a valid license.
After a day's effort of trying different things, such as moving across Sun's JRE providers into IBM JRE's "java.security" file, I ended up decompiling soapUI's code. It appears soapUI's decryption method is "RSA - SunJCE - 512", which requires the "BouncyCastle" security provider. The solution was to add this line to the JRE's java.security file:
security.provider.1=org.bouncycastle.jce.provider.BouncyCastleProvider
Voila, you can now activate your license. Although...

SoapUI Pro 5.1.2 has a gotcha when running testrunner.sh. It will attempt to validate your license as well, and requires you to have X11 forwarding enabled (no matter what). So if you're like me and are running SoapUI Pro on a headless Linux environment, you're stuffed. We ended up downgrading to 5.0.0, where this X11 port forward is not required.

Maximo startup problems!

Our infrastructure utilizes WebSphere MQ as our Maximo queue backend. Our automation framework (consisting mainly of jython scripts injected into wsadmin) sets up CQIN and SEQIN queues, as well as activation specifications and whatnot, with 1 click of a button, so our margin for error is pretty low once it's off the ground running.
So when this error appeared in SystemOut.log on Maximo startup it was quite discomforting:

[8/25/14 15:53:39:542 EST] 000003d9 SystemOut O 25 Aug 2014 15:53:39:510 [ERROR] [MXServer] [] java.lang.NullPointerException at psdi.iface.jms.JMSContQueueProcessor.processMessage(JMSContQueueProcessor.java:253) at psdi.iface.jms.JMSListenerBean.onMessage(JMSListenerBean.java:203) at com.ibm.ejs.container.WASMessageEndpointHandler.invokeJMSMethod(WASMessageEndpointHandler.java:138) at com.ibm.ws.ejbcontainer.mdb.MessageEndpointHandler.invokeMdbMethod(MessageEndpointHandler.java:1146) at com.ibm.ws.ejbcontainer.mdb.MessageEndpointHandler.invoke(MessageEndpointHandler.java:844) at com.sun.proxy.$Proxy33.onMessage(Unknown Source) at com.ibm.mq.connector.inbound.MessageEndpointWrapper.onMessage(MessageEndpointWrapper.java:131) at com.ibm.mq.jms.MQSession$FacadeMessageListener.onMessage(MQSession.java:125) at com.ibm.msg.client.jms.internal.JmsSessionImpl.run(JmsSessionImpl.java:2747) at com.ibm.mq.jms.MQSession.run(MQSession.java:950) at com.ibm.mq.connector.inbound.ASFWorkImpl.doDelivery(ASFWorkImpl.java:88) at com.ibm.mq.connector.inbound.AbstractWorkImpl.run(AbstractWorkImpl.java:216) at com.ibm.ejs.j2c.work.WorkProxy.run(WorkProxy.java:668) at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1862)

This error was flooding the logs every few milliseconds, and causing CPU starvation!
It seemed to suggest the "JMSContQueue" was trying to "processMessages" (duh). The problem: The WebSphere MQ infrastructure (which we didn't own) did not have the queue yet. For some reason Maximo polls infinitely for the queue...so I changed "intjmsact" to point at a queue which did exist, and voila!

But that wasn't the end of it! When the queue was finally created and "intjmsact" was configured back to point at the original queue, same error message!
This time, the problem was that there were messages already on the queue, which Maximo did not recognize. Maximo picked them up, rejected them, and put them back on the queue, causing yet another infinite cycle. Deleting the messages resolved the issue.

Splunk and lookups

The client upgraded Splunk from 5.0.8 to 6.1.2, worthwhile upgrade imho. But it messed up my query, possible bug.

Given this query: (not exact for commerical reasons)

index=prod sourcetype=wps.log module="PXY_*" (`transaction_filter`)
  | dedup host _raw
  | eval timestamps=_time
  | convert timeformat="%s" ctime(_time) as TimeStamp
  | search [| inputlookup outages | eval StartTime = strftime(strptime(Start,"%d/%m/%Y, %H:%M"),"%s")
            | eval EndTime = strftime(strptime(End,"%d/%m/%Y, %H:%M"),"%s")
            | eval search = "(TimeStamp < \""+StartTime+"\" OR TimeStamp > \""+EndTime+"\")"
            | fields search | mvcombine search | eval search = "(" + mvjoin(search, " ") + ")"]

I had used this in v5 to filter out results that fell within an outage period. The pre-req for this is a lookup table called 'outages'.

The result of the subsearch looked like this.

((TimeStamp < "1398949200" OR TimeStamp > "1398974400") (TimeStamp < "1399554000" OR TimeStamp > "1399575600") (TimeStamp < "1399726800" OR TimeStamp > "1399748400") (TimeStamp < "1399986000" OR TimeStamp > "1400011200") (TimeStamp < "1400072400" OR TimeStamp > "1400097600") (TimeStamp < "1400418000" OR TimeStamp > "1400443200") (TimeStamp < "1400504400" OR TimeStamp > "1400529600") (TimeStamp < "1400763600" OR TimeStamp > "1400788800") (TimeStamp < "1400763600" OR TimeStamp > "1400778000") (TimeStamp < "1400936400" OR TimeStamp > "1400958000") (TimeStamp < "1401282000" OR TimeStamp > "1401307200") (TimeStamp < "1401454800" OR TimeStamp > "1401516000") (TimeStamp < "1401541200" ))

Before the upgrade, it just worked as it should've. After upgrade, nada. Defect perhaps?

Websphere Messaging Engine not starting

In trying to automate WebSphere installation, we ran into the titled problem.
As with my other posts, we've got corporate DBAs who we engage to create user accounts and databases for us. Our initial guess was the user account we had created for us didn't have the right privileges, but there were no SQL exceptions in FFDCs. When starting the messaging engine, we'd get this error:

The messaging engine "ME_name" cannot be started as there is no runtime initialized for it yet, retry the operation once it has initialized. For the runtime to successfully initialize the hosting server must be started, have its 'SIB service' already enabled, and dynamic configuration reload enabled. If this is a newly configured messaging engine and it is the first messaging engine to be hosted on this server, then it is most likely the 'SIB service' was not previously enabled and thus the server will need to be restarted. The messaging engine runtime might not be initializing because of an error while trying to start, examine the SystemOut.log of the hosting server to check for error messages indicating the problem

The node server SystemOut.log revealed pretty much nothing. The nodeagent had a number of FFDCs. So I thought perhaps it was a firewall problem, was on the right track...
We found:

• port 9420 was new to us. We were used to WebSphere v7, and looking through serverindex.xml we noticed a port called Status Update Listener: More info
• netstat on the node server and all the ports listening were not matching what we got opened through firewalls. So we changed them.
• the FFDCs had an "UnknownHostException: *". The application server wasn't starting properly either, so this error pointed me in the right direction. The host needs to be defined for at least the SOAP_CONNECTOR_ADDRESS, and IPC Connector port we set to localhost
• I got the messaging engine running by setting the schema (under Bus > Messaging Engine > Message Store > Schema) and the user to the same value.