Wednesday, December 25, 2019

Adding ActiveDirectory users to Jenkins

I work(ed) at a Windows-centric organization. Jenkins running on Windows can cause quite a stir. When adding AD users to Jenkins under Configure Global Security, the first problem you’ll encounter is, in Jenkins, users are case-sensitive. That means if your AD user is JOHN, you’ll need to add both JOHN and john, otherwise when the user decides to login with small case, it won’t work. A bigger problem is, once you cross about 50 users, you’ll start getting exception as documented in JENKINS-26963 - Form too large 213549>200000. The quick fix is to add a JVM parameter to jenkins.xml. If you’re running Jenkins behind jetty on Windows, you actually need to do this instead: prunmgr.exe //ES//Jenkins (You can get the edit string from Windows services) Funnily enough, if you visit https://wiki.jenkins.io/display/JENKINS/Jetty, the lone comment on that page addresses the above. Considering Jenkins is bundled on Jetty you would think this was better documented.

Friday, December 20, 2019

Mono-repo with Lerna

Dev team wanted to use lerna - defined bootstrap and postinstall commands that called lerna. Lerna bootstrap default behaviour uses npm ci This will fail if no package-lock.json. If you set postinstall this will never exist Package-lock not sustainable in git when many developers contributing code Thus will fail with nipm package-lock doesnt exist. We had to use —no-ci and sacrifice the speed boost. Lerna bootstrap will also run forever if downstream npm install demands output. For eg in my case semantic.json had backslashes for paths, and gulp returns a prompt - semantic.json exists do you want to Skip Install. Lerna with loglevel silly will get stuck at ‘npm install’ on the leaf and not say anything. I found advise about not putting lerna bootstrap in a postinstall command, however that is no longer applicable. Lerna publish creates git tags for every subpackage but only if its changed, so you end up with a mess of tags with different versions. So to make sure we have 1 version for everything in the repository, I use sed to replace version in the root and packages package.json, but also change the versions of local dependencies. Lerna bootstrap calls node-gyp rebuild, which must connect to internet to download node. Unless you set nodeconfig to a local installation. Easiest way was to set config in .npmrc. Since lerna uses webpack, I got away with installing this globally. For gulp, even if I installed globally, it still complained the command didn’t exist. The solution was to —save-dev and make it a devDependency. Some places actually suggest to not use lerna bootstrap and switch to using file specifiers for local dependencies. This ends up a greater headache - lerna bootstrap downloads a ton of gulp dependencies (gulp-help, gulp-concat, etc.). Don’t heed that advice.

Sunday, November 3, 2019

Generating SHA-256 checksums for Maven artifacts

This one is thoroughly undocumented. Didn't go through the plugins code to work this one out, it worked purely by chance...

My organization requires that for all artifacts to be released, a SHA-25 checksum needs to be generated. I've standardized on pom.xml for all projects in order to upload artifacts to Nexus. My alternative was to upload artifacts via Jenkins in a pipeline using the Nexus uploader block, however it doesn't seem there's a simple way to get identify ahead of time the artifacts that would be built in the Maven dependency tree. If there was, I could just run "sha256sum" on this list... I did try suggestion from here but if I recall it didn't list artifacts from child modules: https://stackoverflow.com/questions/36936238/create-a-list-of-artifacts-that-are-build-by-a-maven-project

Far easier to use a Maven plugin with "mvn deploy". Cue the checksum-maven-plugin:
https://checksum-maven-plugin.nicoulaj.net/examples/generating-project-artifacts-checksums.html

Looks simple enough. Quote: "This configuration will generate checksum digest files for the project main and attached artifacts".

      <plugin>
        <groupId>net.nicoulaj.maven.plugins</groupId>
        <artifactId>checksum-maven-plugin</artifactId>
        <version>1.8</version>
        <executions>
          <execution>
            <goals>
              <goal>artifacts</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <!-- put your configurations here -->
        </configuration>
      </plugin>
When I tried this plugin, no SHA-256 checksum gets generated.

Within the responses on the Github repository for that plugin, I find this:
https://github.com/nicoulaj/checksum-maven-plugin/issues/39


<plugin>
    <groupId>net.ju-n.maven.plugins</groupId>
    <artifactId>checksum-maven-plugin</artifactId>
    <version>1.3</version>
    <executions>                   
        <execution>
            <id>checksum-artifacts</id>
            <phase>package</phase>
            <goals>
                <goal>artifacts</goal>
            </goals>
            <configuration>
                <csvSummary>false</csvSummary>
                <shasumSummary>true</shasumSummary>
                <shasumSummaryFile>sha512-libs.sum/shasumSummaryFile>
                <individualFiles>false</individualFiles>
                <algorithms>
                    <algorithm>SHA-512</algorithm>
                </algorithms>
                <types>
                    <type>jar</type>
                </types>
                <scopes>
                    <scope>runtime</scope>
                </scopes>
            </configuration>
        </execution>
    </executions>
</plugin>
I guess it used to have a different group name prior to version 1.5, however this version of the plugin was at least printing a line in the Maven output indicating this plugin was getting invoked. I thought my plugin wasn't even getting called! However still no checksum was getting generated.

I'd almost given up, until my colleague started using the plugin and SHA-256s were getting generated and auto-uploaded to Nexus. After a bit of digging, I found that only artifacts in the "${workspace}/target" directory were getting checksums generated. My artifacts were getting generated in child module folders, and any arbitrary directory the project called for - e.g. from using maven exec plugin or antrun.

The solution was to add an extra antrun step to move any built artifact into the project root's target directory, and then use attach-artifacts to include it for upload to Nexus. In some of my cases, I had to add an extra module to perform this after other child module's had completed building. Sometimes the Maven reactor wouldn't order the child module's properly, especially if the module's were built with proprietary Maven plugins (e.g. Temenos products). Not the user-friendly experience I expected for generating checksums!