Alerting on JIRA Problems using Nagios

So I ran into an interesting situation this past Monday. Apparently my Primary DNS had been down for at least a week. I went to go look at my network monitoring tool (LibreNMS) – and THAT was down too – for what I can guess is at least two weeks! Granted I haven’t been doing as much on my Homelab since early March when I went into the hospital, this was still not a good state of affairs.

So I decided to stand up a Nagios instance to monitor and alert when I have critical systems down. After getting it stood up, it didn’t take me long to start thinking about how I could use this with JIRA, which is now the topic we are going to cover today!

A bit of history

As you know, when I started my Atlassian journey, I was in charge of more than just JIRA. Nagios was one of my boxes I inherited as well. So I’m somewhat familiar with the tool already and how to configure it. I’ve had to modify things during that time, but never do a full setup. However, I knew I wanted to do more than monitor if JIRA was listening to web-traffic. So as part of the whole installation, I decided to dive in and see what she can do.

How to select what to Alert on.

Selecting what I want to be alerted for has always been a balancing act for me. You don’t want to have so many emails that they become worthless, but you don’t want to have so few that you won’t be alerted to a real problem.  

The goal of alerting is to clue you into problems so you can be proactive. Fix back end problems before they become a user ticket. So I always try to take the approach “What does a user care about?”

They care that the system is up and accessible, so I always monitor the service ports, including my access port. So that’s three.

A user also cares that their integrations work. If your integrations depend on SSL, and your clock drifts too far out of alignment, those integrations can fail – so I want to check the system is in sync with the NTP Server.

A feature that users love is the ability to attach files to issues. This feature will eventually chew up your disk space, so I’ll also want to monitor the disk JIRA’s home directory lives on. 

Considering I’m using a proxy, I’ll want to be sure the JVM itself is up, so I’ll need to look at that. I’ll also want to be sure that JIRA is performing at it’s best, and isn’t taking too long to respond, so I’ll want an alert for that as well.

Do you see what I’m doing? I’m looking at what can go wrong with JIRA when I’m not looking and setting up alerts for those. The idea here is I care about what my users care about, so I want the Nagios to tell me what is wrong before my users get a chance to.

So…configurations.  

Now comes the fun part. Nagios’ configuration files is a bit much to take in at first. However, I will be isolating the Atlassian specific configurations to make things a bit easier on all of us. First, lets start with some new commands I had to add.

###############################################################################
# atlassian_commands.cfg
#
#
# NOTES: This config file provides you with some commands tailored to monitoring
#        JIRA nodes from Nagios
# AUTHOR: Rodney Nissen <rnissen@thejiraguy.com
#
###############################################################################


define command {
    command_name    check_jira_status
    command_line	$USER1$/check_http -S -H $HOSTADDRESS$ -u /status -s '{"state":"RUNNING"}'
	}
	
define command {
    command_name	check_jira_restapi
	command_line	$USER1$/check_http -S -H $HOSTADDRESS$ -u /rest/api/latest/issue/$ARG3$ -s "$ARG3$" -k 'Authorization: Basic $ARG4$' -w $ARG1$ -c $ARG2$
	}
    
define command {
    command_name    check_jira_disk
    command_line    $USER1$/check_by_ssh -H $HOSTADDRESS$ -l nagios -C "/usr/lib64/nagios/plugins/check_disk -w $ARG2$ -c $ARG3$ -p $ARG1$"
    }
    
define command {
    command_name    check_jira_load
    command_line    $USER1$/check_by_ssh -H $HOSTADDRESS$ -C "/usr/lib64/nagios/plugins/check_load -w $ARG1$ -c $ARG2$" -l nagios

The first two commands here are VERY tailored to JIRA. The first one checks that the JVM is running, all with a handy HTTP request. If you go to your JIRA instance and go to the /status directory, the JVM will respond with a simple JSON telling you the state of the node. You use this feature in JIRA Data Center, so your load balancer can determine which nodes are up and ready for traffic. Buuut…it’s on JIRA Server too, and we can use it for active monitoring. So I did. If JIRA returns anything other than {“state”:”RUNNING”}, the check will fail and you will get an alert.

The second is a check on the rest API. This one will exercise your JIRA instance to make sure it’s working without too much load time for users. The idea here will search for a known issue key, and see if it returns valid information within a reasonable time. $ARG1$ is how long JIRA has before Nagios will issue a warning that it’s too slow (in seconds), and $ARG2$ is how long JIRA has before Nagios considers it a critical problem. $ARG3$ is your known good Issuekey. $ARG4$ is a set of credentials for JIRA encoded in Base64. If you are not comfortable just leaving your actual credentials encoded as such, I’d suggest you check out the API Token Authentication App for JIRA. Using the App will allow you to use a token for authentication and not expose your password.

The third commands here are for checking JIRA’s home directory ($ARG1). $ARG2$ and $ARG3$ are percentages for the warning and critical thresholds, respectively.

The fourth is for checking the system load. This one is relatively straight forward. $ARG1$ is the system load that will trigger a warning, and $ARG2$ is the system load that shows you have a problem.

Now for the JIRA host configuration:

###############################################################################
# jira.CFG - SAMPLE OBJECT CONFIG FILE FOR MONITORING THIS MACHINE
#
#
# NOTE: This config file is intended to serve as an *extremely* simple
#       example of how you can create configuration entries to monitor
#       the local (Linux) machine.
#
###############################################################################



###############################################################################
#
# HOST DEFINITION
#
###############################################################################

# Define a host for the local machine

define host {

    use                     linux-server            ; Name of host template to use
                                                    ; This host definition will inherit all variables that are defined
                                                    ; in (or inherited by) the linux-server host template definition.
    host_name               jira
    alias                   JIRA
    address                 192.168.XXX.XXX
}


###############################################################################
#
# SERVICE DEFINITIONS
#
###############################################################################

# Define a service to "ping" the local machine

define service {

    use                     generic-service           ; Name of service template to use
    host_name               jira
    service_description     PING
    check_command           check_ping!100.0,20%!500.0,60%
}


# Define a service to check SSH on the local machine.
# Disable notifications for this service by default, as not all users may have SSH enabled.

define service {

    use                     generic-service           ; Name of service template to use
    host_name               jira
    service_description     SSH
    check_command           check_ssh
}



# Define a service to check HTTP on the local machine.
# Disable notifications for this service by default, as not all users may have HTTP enabled.

define service {

    use                     generic-service           ; Name of service template to use
    host_name               jira
    service_description     HTTP
    check_command           check_http
}

define service {

    use                     generic-service
    host_name               jira.folden-nissen.com
    service_description     HTTPS
    check_command           check_https
}


define service {
    use                     generic-service
    host_name               jira
    service_description     NTP
    check_command           check_ntp!0.5!1
}

define service {
	use				generic-service
	host_name			jira
	service_description		JIRA Status
	check_command			check_jira_status
	}
	
define service {
	use				generic-service
	host_name			jira
	service_description		JIRA API Time
	check_command			check_jira_restapi!2!3!HL-1!<Base64 Credentials>
	}
    
define service {
	use				generic-service
	host_name			jira
	service_description		JIRA System Load
	check_command			check_jira_load!5.0,4.0,3.0!10.0,6.0,4.0
	}
    
define service {
	use				generic-service
	host_name			jira
	service_description		JIRA Home Directory Free Space
	check_command			check_jira_disk!<JIRA Home>!75%!85%
	}

So, first, we define the host. This section is information specific to JIRA. Then we start setting services for JIRA. Within Nagios, a Service is a particular check you want to run.

The next four options are pretty standard. These are checking Ping, the two service ports (HTTP and HTTPS), and the SSH port. The SSH Port and HTTP/S port checks will also check that those services are responding as expected.

The next check is for NTP. I have this setup to warn me if the clock is a half-second off and give me a critical error if the clock is off by one second. These settings might be too strict, but it has yet to alert, so I think I have dialed it in well enough.

The next is the JIRA Status check. This service will check /status, as we mentioned earlier. It’s either the string we are expecting, or it’s not, so no arguments needed.

After that is my JIRA API Check, which I set up to check the HL-1 issue. If the API Call takes longer to 2 seconds, issue a warning, and if it takes longer than 3 seconds, Nagios issues a critical problem. This alert won’t tell me exactly what’s wrong, but it will tell me if there is a problem anywhere in the system, so I think it’s a good check.

The last two services are systems check – checking the System Load and JIRA home directory disk, respectively. The Load I haven’t had a chance to dial in yet, so I might have it set too high, but I’m going to leave it for now. As for the Disk check, I like to have plenty of warning I am approaching a full disk to give me time to resolve it, so these numbers are good.

The last step is to add these to the nagios.cfg file so that they get loaded into memory. However, this is as easy as adding the following lines into the cfg file.

# Definitions for JIRA Monitoring
# Commands:
cfg_file=/usr/local/nagios/etc/objects/atlassian_command.cfg

# JIRA Nodes:
cfg_file=/usr/local/nagios/etc/objects/jira.cfg

And that’s it! Restart Nagios and you will see your new host and service checks come up!

Nagios in action.

So I’ve had this configuration in place for about a day now, and it appears to be working. The API Time check did go off once, but I did restart the JIRA Server to adjust some specs on the VM, so I expected the delay. So I hope this helps you as you are setting up alerts for your JIRA system!

And that’s it for this week!

We did get a bit of bad news about Summit 2021 last week. Out of an abundance of caution, Atlassian decided to go ahead and make all in-person events of 2020 and Summit 2021 virtual events. However, they have almost a year to prepare for a virtual Summit – as opposed to the 28 days they had this year. So I am excited to see what ideas Atlassian has to make this a fantastic event!

Don’t forget our poll! I’m going to let it run another week!

Don’t forget you can check me out on Twitter! I’ll be posting news, events, and thoughts there, and would love to interact with everyone! If you found this article helpful or insightful, please leave a comment and let me know! A comment and like on this post in LinkedIn will also help spread the word and help others discover the blog! Also, If you like this content and would like it delivered directly to your inbox, sign up below!

But until next time, my name is Rodney, asking “Have you updated your JIRA Issues today?”

Dragon Slayer Part 2: The Revamp

Well, I’ve finished the revamp! It took me a little longer than expected, but here we are.

Of course, you guys enjoyed last week’s article on Automation for JIRA. While I would not like it to be the sole focus of the blog, if you guys bring me good ideas for potential Automations, I would love to do more of those kinds of pieces. But that’s not why we are here today.

So in total, I reworked all nine pages, each about as long as a typical blog post. Today I’d like to go into some thoughts, tricks, and pitfalls to be aware of if you try to do it from my instructions.

Time is not on your side.

So, to be clear, the last five modules all depend on a Mercurial Repo hosted on Bitbucket Cloud. It’s something that works well in this situation – except for the following complication. 

Atlassian has announced they are ending support in Bitbucket Cloud for Mercurial, with all existing Mercurial Repos disappearing on July 1st, which leaves the Dragon Slayer Challenge in a bit of a pickle.

As of right now, I intend to monitor it to see if they update their instructions with a git repository. If that does not happen, I’ll be taking down my instructions on July 1st as well. It only seems right, honestly. I would not like to be answering comments for the next few years that everything past Stage 5 isn’t working.

So if you are looking to try this yourself – do it now!

Fish eye gadgets are still…off.

So, I spent half a day debugging this – only to get nowhere. There are a few answers out there for the problem, but they are six years old. To further complicate things, my testing proved that their problem back then was not my problem today.

What problem is that? Simple, I could not get a Fisheye Gadget to return a valid Repository no matter what I tried. Every time I’d get the same, “The specified FishEye repository is not configured” – which was malarkey. The Repositories would work correctly within the JIRA Development Panel – just not the gadgets. Upping the logging on the FishEye plugin revealed nothing of usefulness in the logs either.

At a certain point, I just had to decide I wasn’t the one to fix this issue. However, if you do have a fix, please PLEASE let me know!

A few last notes

I tried to go through and click every link, touch every instruction, and redo every screenshot. But there may be parts I missed. If you notice something that isn’t working, please let me know so I can update it!

There are points where things weren’t working as described, and I had to find out why and fix it. Every time this did happen, though, I did go back to the appropriate place and update the instructions. I feel this is an improvement on the old instructions from Atlassian and is something I am happy to put out.

That being said, this is Atlassian’s work originally. If they ask that I take it down, I will do so without warning to you guys. I don’t want to bite the hand that feeds me. I honestly don’t think it will come to that, but I figured I’d give you fair warning.

So, Get on with it!

You can find the start of the journey here. If you’ve never messed with the system side of running the Atlassian Stack yourself, it’s a great place to get started.

I’ve spent a good bit of time on this update, so if you appreciate it, please give it a like and cI’ve spent a good bit of time on this update, so if you appreciate it, please give it a like or comment on LinkedIn or a retweet on Twitter. That’s all it takes to help others discover this blog! You can also subscribe to the blog using the form below to get new posts directly to your inbox. Don’t forget to follow me on Twitter at @theJIRAguy.

Also, a discussion at work got me curious. How do you spell it: Subtask or Sub-task? Answer below!

But until next time, my name is Rodney, asking “Have you updated your JIRA issues today?”

Dragon Slayer 2020 – Pt. 1

What’s up, everyone? It’s been a busy week for me. However, in the “excellent things” department, I passed my exam!

So shiny…

That means I’m exactly one Cert away from Atlassian Certified Master status. And I think we all know that my next certification exam is the ACP-200.

Revenge is needed….

However, that is going to have to wait. I am not losing to that exam, so study time is needed. No, this week is going to be a bit of a throw-back. I’m going to be attempting the Atlassian Dragon Slayer challenge to see how it holds up. This challenge is to set up a fully integrated development suite using JIRA, Confluence, Fisheye, Crucible, and Bamboo. Considering I already have a test JIRA instance, I’m going to follow the directions linked.

Next week for part 2, I’m going to re-write it with modern JIRA, Confluence, Bamboo, and Bitbucket in mind so that if you want to follow this old quest with modern tools, you can. So, let’s not dally any longer.

Dragons with JIRA Stage 1 – Set Up Environment and JIRA

As my JIRA Instance is already running and relatively up to date, I can skip steps 1-4 and go directly to Step 5.

Anyone else remember when Atlassian version numbers would get that high? Needless to say, I think we’re over the required version.

All told, Step 5 is mostly setting up users and groups for the other system. Step 6 is setting up the project and Dashboard. The only question I ran into here is the project type. The instructions ask for a “Software Development’ Project,” however, this was before JIRA 7, where JIRA Agile became JIRA Software (with three different types of Software Projects). I opted considering the criteria to go with a “Basic software development” project – as I felt that most closely followed the instructions so far. That may come to bite me in a moment, as it’s time to move onto the next Stage.

Dragons Stage 2 – JIRA Add-Ons

Well, that whole JIRA Agile/Software thing ended up not being a problem, as the first step on the next page asks me to set up a new project and board. That is one thing I’ll give Atlassian – this kind of deal is A LOT easier than it used to be. The instructions have you create the project, then the board. You can now do this all in one go in JIRA Software. So, create the Scrum project, create the bugs they specify, create/start a sprint, and I get this.

The next section for this Stage involves setting up Capture for JIRA by Zephyr Smartbear. This App lets you capture screenshots of web pages (among other things). This tool is great for capturing more information on bugs, which is how the challenge has us use it

Dragons Stage 3 – Install Confluence

So, Stage 3 is installing Confluence. And well…

…I already have a Confluence.

I still needed to do some basic configuration and setup work here. I added JIRA as a directory in Confluence (honestly, it’s easier than the OpenLDAP config I had going), and created the DRA Space and started modifying its home page. And this was where I ran into trouble.

This…this innocuous looking line from my Apache config cost me 2 hours worth of time to fThis innocuous-looking line from my Apache config cost me 2 hours worth of time to figure out. This setting would unset the authentication every time Confluence tried to talk to JIRA, which meant the JIRA Issues/Filters Gadget would not work. The most infuriating part – there was no error in either Confluence’s or JIRA’s log. I only figured it out as I happened upon the right KB Article on it…

However, with that fixed, I finally got everything up and running normally, and was able to finish Stage 3.

Dragons Stage 4 – Install Team Calendars in Confluence

Team Calendars – now this one I know just works. I install it per the usual method (I am certainly burning through the free trials today). Honestly, this one just worked pretty straight forward – follow the instructions here.

Dragons Stage 5 – Install FishEye and Crucible

Stage 5 is installing Fisheye – and honestly, this has not been something I’ve done since early in my Atlassian Administrator career. However, the instructions included seemed a bit dated. So I instead popped over to the Atlassian KB and found this article on Installing Fisheye on Linux and Mac. You have to love it when you find the article you need. Then it was a matter of connecting my internal bitbucket server instance (for thoroughness), as well as the remote Bitbucket repo mentioned in the Stage 5 instructions, and we’re off to the races!

Dragons Stage 6 – Get JIRA and FishEye Talking

So Step one is setting the link between the Repo from Stage 5 and the DRA project in JIRA. It’s relatively simple to set up. It was at this point that I had to figure out why the “Source” tab they mentioned wasn’t showing up. Then I remembered that JIRA had moved this to the Development Panel some time ago.

“It’s fantastic when stuff just works,” is what I’d be saying if not for Step 3. No matter what I did, I could not get the gadget to work. The gadget would keep saying, “The Repo was not configured.” I knew the Repo had been configured properly. Other Fisheye Gadgets worked with the Repo, just not this one. I need to move on with this one.

Dragons Stage 7 – Get JIRA and Crucible Talking

This one was fairly straight forward. I linked a Crucible Project to the DRA JIRA Project and went through the process of creating a review, finding a defect, creating a JIRA issue from it, and resolving it. The only problem I had was when I couldn’t find the “Create Issue” link – but that one was purely an ID-10T error on my part. Another one down!


Dragons Stage 8 – Install Bamboo

I can see the light at the end of the tunnel – but this one is another install challenge. I I can see the light at the end of the tunnel – but this one is another install challenge. I overbuilt the Fisheye server also to host Bamboo as both are temporary in my setup. This one also needed some updating, so I followed the more up-to-date KB Article Installing Bamboo on Linux. Afterward, I set up Bamboo to use JIRA for Authentication and double-checked that the groups were set up. 

Now it was time to get all the application links in place. With five applications now, this process is getting a bit long, but not painful.

Now it was time to configure the actual build. This one took me a second as I had to figure out where the tools directory was on the host system ( /usr/share/maven/ if you are wondering), but I got it set up, debugged, and a successful build, which completed the next to the last Stage!

Dragons Stage 9 – Bamboo Gadgets and JIRA Victory

So last stage – looks like it’s more gadgets time. And…there’s a problem. When I go to add So last Stage – looks like it’s more gadgets time. And there’s a problem. When I go to add the gadgets they request, I can see and add them to the Dashboard. But then they don’t load their settings panel at all.

It turns out – it was my browser. Because I’m not bothering to put these behind an SSL proxy, Firefox did not like that. Disable the protection (temporarily), and they work!

Final thoughts

So, here’s the deal. These guides were last updated in 2017, and based on the repo data, written much earlier. May and June may be our last chance to run this for a bit, as Atlassian is removing Mercurial Ropes from Bitbucket Cloud.

However, this was a fun little run-through of all the various bits of the Atlassian Development Stack. I got to touch several products that I haven’t looked at in years and got some practical experience in installing them from scratch. It’d be interesting to do a speed run of these to see how long it would take.

So now comes the fun part – updating this with new instructions and getting it all working on the modern versions of the tools! If you want to check that out, be sure to sign up to receive emails from the blog. You can also follow me on Twitter at @theJIRAguy. But until next time, my name is Rodney, asking, “Have you updated your JIRA Issues today?”

JQL – How have we not talked about this already?

Hello everyone! How has your week been going? This week we are going to talk about JQL – and honestly, I’m not entirely sure how this hasn’t come up already. I know I’ve provided a few queries here and there to help you in other topics, but today I’ll be talking about some of my favorite tips and tricks to get the most out of your queries. So without any more delays, let’s get into it.

Always about the Docs

So it’s been a while since I’ve included this section – but I feel this subject could be helped with some further reading. So for this, I am referencing two documents:

I should note that the function reference is for JIRA Cloud – so it does have some new functions that have not made it to server. However, the options I’m covering here will work for JIRA Cloud, JIRA Server, and JIRA Data Center.

Currentuser()

So, this first trick is one of my favorites, as it allows you to customize a query to be specific for whoever is viewing it. In your query, you can set a user field equal to currentuser(), and the function will return whoever is logged in and viewing the query. For example, let’s take the query:

assignee = currentuser()

And let us take a look at what that looks like when running from the Admin account, and my account:

Each one returns whatever is assigned to the specific user. What’s more, this would carry over to any dashboard that uses this query – allowing you to customize a single dashboard to whoever views it.

This also works with any user field: reporter, assignee, watcher – and even custom fields. This is a handy trick to have if you don’t like to spend your day making queries and dashboards…I mean, who would enjoy that, am I right…

Working with time

So, when I’m giving some training for JQL, the most common problem people have is working with time. “How do I search for stuff in the past?” “How do I work with time?”…it seems to be a challenge.

Here’s a trick that’s always helped me. Most databases store time in the Unix Epoch time format – which is simply counting the seconds between midnight of 1 Jan 1970 and today. This means that any time in the past will be less than the current time, and any time greater than now will be in the future. Same for JQL. The past is “less than” the timestamp you are looking at, and the future is “greater than”.

However, JIRA also gives us some handy functions so we are not having to spell out particular date. First off is now(), which simply returns the current timestamp whenever it’s run. Handy when just trying to find out what’s past due, such as in this example (courtesy Atlassian):

duedate < now() and status not in (closed, resolved)

In the above query, duedate < now() returns everything that has a duedate in the past, and status not in (closed, resolved) returns everything that is currently open – giving you an idea what is past-due.

DayWeekMonthYear
startingstartOfDay()startofWeek()startaOfMonth()startOfYear()
endingendOfDay()endOfWeek()endOfMonth()endOfYear()

These are great for establishing a time frame. The really powerful thing about the functions on this table is they take parameters, meaning you can adjust them forward and backward – to say find all the issues resolved last month:

resolutiondate > startOfMonth(-1)  AND resolutiondate < endOfMonth(-1) 

In this query, we shift both startOfMonth() and endOfMonth() by -1. The minus (-) tells us it’s a previous time, and the one (1) tells us to shift one whole month, meaning both of these times return values for the previous calendar month. This gives us a handy time-box for our queries – which is great for a report.

Sprints and Versions

So, this section is for my JIRA Software teams out there.

When I’m a part of Agile teams, I like to have a dashboard we can all view that tells us what’s going on in the current sprint. For this, we can use the opensprints() function. I like to pair this with a project key query so that I only get the open sprint for my project, such as:

Project = "Project Name" and Sprint in openSprints()
Pardon the blur…I actually don’t have any open Sprints in my test instance, so I had to use my work instance.

A note on openSprints() is that it only supports the operators IN and NOT IN. Using an equal sign will cause the query to return an error.

There is also the operators futureSprints() and closedSprints(). However, these return ALL future and closed sprints, respectively, so further filtering will be needed to focus on any particular sprint (such as Sprint = “Name”)

So, let us say you are in a situation where you care more about what has and hasn’t been released than what is in any particular sprint. Then you’ll be interested in the functions latestReleasedVersions() and earliestUnreleasedVersions(). These operate on the fields AffectedVersion, FixVersion, as well as any custom fields with type Version. Let’s say you want to know what bug-fixes have gone out the door:

issuetype = Bug and fixVersion = latestReleasedVersion()

The neat trick about this query is that it will automatically update if your team releases a new version, meaning if you use it in a dashboard, it will always show the bugs fixed in the last released version.

Filtering out sub-tasks

So this one comes up often. A manager wants a dashboard, but they don’t want to get bogged down in the minutia of the sub-tasks. This a handy trick I use to filter out all sub-tasks from a query. I simply and the existing query with this clause:

issuetype not in subtaskIssueTypes()

It’s a simple thing to add but is an easy way to filter out all sub-tasks. Because of how it works, if you add any new sub-task types to the project in the future, the filter will still work without needing to be updated.

Cleaning up watched issues

So, JIRA in its current form sends out A LOT of emails.

Just a *few* emails…

Atlassian has heard our cries, and they are working on it, but in the meantime – this is where we are. And as an Admin, you will be touching a lot of issues – which will put you on their watchers list automatically. So I find it helpful to run this JQL and clean things up every few months.

watcher = currentuser() and project not in ("Home Project A", "Home Project B", ...)

This gives me a list of all issues I’m watching on an instance that is outside of what I consider to be my “Home Projects” I use to organize my work. From there you can run a bulk change on the list, and on Step 2, choose the “Stop Watching Issues” operation. I don’t know if this honestly counts as a JQL trick, but it has helped keep me sane many a time.

And that’s all for this time!

Did you learn anything new? What are some of your favorite JQL tricks that I didn’t cover? If I get enough I’ll do a part 2 to this!

So it’s been a busy time for me. I decided to take advantage of the current time I have to study for my next ACP exam. It’s scheduled for May 8th, which doesn’t give me too much study time left. But hopefully, I’ll be sending out good news the following week!

My Wife and I also have been working on 3D printing face-shields for first responders – you can read about our efforts (okay, mostly her efforts) on her blog here: https://cadstories.com/2020/04/17/engineering-and-covid-19/

I also have a twitter account for the blog now! If that’s your preferred source of Social Media, give me a follow! I’ll be posting blog posts there, as well as anything else Atlassian related I find interesting! https://twitter.com/theJIRAguy You can also sign up here to get an email every time I publish a new blog post! Simply use the sign-up form at the bottom of this post!

Until next time though, this is Rodney, asking “Have you updated your JIRA Issues today?”

App Review: VisualScript for JIRA

Hey everyone! Hope everyone is staying safe and healthy this week. I normally work from home, so it’s been mostly business as normal for me. The only change is now my wife is also working from home. However, Animal Crossing has been doing a lot to help us keep busy. In speaking of which, I did catch this:

Even in game, it’s all about that Atlassian life…

So this week we have another App Review. You guys seemed to like the last App Review I did, so I figured I’d look at another. Fortunately, the time is right for looking at new Apps. Traditionally Summit is where various Atlassian Partners would be putting out their Apps for review and testing. I was fortunate enough to connect with Evan Golden with SmartDraw, who gave me the grand tour.

Now, I always feel it’s important you know where my motivations are. As such, this is not a paid sponsorship. This is a review done on my own volition, because I feel it does solve a problem.

The Problem

It is my believe that every App you add into JIRA should solve some problem that JIRA Alone cannot solve. Otherwise, why are you wasting the money, time, and resources to run it? So, lets imagine this:

You are in a meeting with a VP who wants to bring some process into JIRA “to improve visibility”. He has no interest in learning to query JIRA or setup a dashboard himself, so he’s unfamiliar with what you can put onto a dashboard. However, he wants a bunch of metrics that JIRA simply don’t have gadgets to do on a dashboard in vanilla JIRA.

You can go out and purchase a bunch of add-ons to do what he wants, but at one add-on per gadget, that can add up quickly – in terms of cost and resources. Not an ideal solution. If only there was one App that could give you any gadget you want with minimal effort…

The Solution

As I hinted to, I don’t like it when an App only adds one dashboard gadget. You can get a tool that does only one thing, and sometimes that is unavoidable, but wherever possible you should get a tool that can be used in a multitude of situations.

That gets us to VisualScript for JIRA. What this App does is allow you to setup custom gadgets (called Reports) powered by JavaScript. This allows you to offload the processing to generate the tables, charts, and figures to your user’s browser, meaning you can get some fancy affects without too much of a hit on performance.

All that’s great and all, but you (like me), probably don’t know java script, nor do you have the time and energy to learn it from scratch. Correct?

That’s where I think this App really shines. It comes with a number of built in reports that you can import and use out of the box, or even modify to suite your needs. These reports span both ITSM and SAFe Agile practices.

ITSM

Most of these appear to not be in the release I have for VisualScript, but they were so compelling for ITSM that I felt I should share them. The version Evan demonstrated for me was a per-release version, and he kindly send me some of the slides to include in the review for me to show to you.

This first report is something I REALLY like. It is simply a report showing the timeline of a problem – from initial incident to Dev story resolution. This is similar to what Atlassian is rolling out for JIRA Service Desk and Opsgenie in the cloud, but you can get this in your JIRA Server/DC instance today!

Another ITSM gadget I really like is these SLA Gauges. I love the visual appearance of a gauge, and feel it can tell you a good bit of information intuitively. Unfortunately, there is no such gadget within JIRA out of the box, and I’ve never been comfortable with one-shot Apps that add them, so seeing this in the demo was pure 😍.

In speaking of SLA’s, getting aggregations of that data can be troublesome. Yeah, you can see which ones have breached, but how is your team doing as a whole. There was a report for that too:

SAFe

As I’ve tried to be clear with everyone, I don’t know everything. While I am traditional agile trained, I’ve never taken it the step further to learning SAFe. I know the general idea, but not the details. That being said, I know what JIRA can do and can’t do, and know some of these next few reports are sorely lacking in JIRA. Disclaimer out of the way, lets dig in.

The first report I thought was interesting was the PI Planning Board.

This gives you a great way of seeing how the stories (and more importantly the dependencies) map across multiple teams across your entire PI. I know this type of view is not really possible in vanilla JIRA, so I can see how if you are practicing SAFe methodology, this can absolutely be needed.

Another great view into your SAFe methodology was the Program Velocity Report. I thought this was a great way to view the entire organization’s velocity to make sure you are meeting your goals.

And yet another view on how your doing is the Epic Dependency Report. This one rather than looking at an org or a group, looks at an individual Epic, and how each of the issues under it are interdependent. This is great for trying to find the critical path of that epic.

And more!

As I said, the built-in scripts are crazy powerful, and WELL worth the price of admission. But I still feel the ability to create your own reports for dashboards is what takes it to the next level. Especially if you have someone who knows JavaScript on staff already.

But even if you don’t, It looks like there is a community where people can share Reports and scripts they have created. This is something I’m most definitely going to keep an eye on in the future, as I’d like to see what people create using this tool.

So, what do you think?

I’m always on the lookout for interesting Add-ons and Apps to share with you. What are some of your favorite Apps for JIRA or Confluence? Let me know some of your favorites here, on LinkedIn, or on Twitter, and if I see something I find interesting, I might cover it!

I’m also always willing to take on reader requested topics, so if you have something you want me to cover, let me know and I’ll look into it!

As I stated last week, I’ve created a twitter account for the blog! If that’s your social media of choice, give us a follow at https://twitter.com/theJIRAguy

But until next time, my name is Rodney, asking “Have you updated your JIRA Issue today?”

Its all about them schemes

Last week we took a look at cleaning up your fields to help improve your system performance. So now we move on to our next Atlassian TAM suggested topic: How to standardize workflows and screens. But before we dive in…

Last week was amazing. I had a lot of engagement on this topic, and even had two more people come forward with requests for more topics to write about. I’m always up to do a reader suggested topic, so if you have something you want to see covered, let me know! It may be a few weeks before I can write on your topic (For once, it’s nice to have a backlog!), but I’ll definitely get to it. You can use the contact form on the blog’s website, or email me directly at contact@thejiraguy.com.

Anyways, lets get into it.

So, let me ask: Why would you want to standardize?

Fair question. This is going to be a lot of work, negotiation, and may come down to you telling them this is how it’s going to be. Why bother?

What’s the benefit that will make all this worthwhile? Well, the first benefit it helps teams within your organization work better together. If a person has to move from Team A to Team 1, they don’t have to relearn how to use JIRA. They’ll see the same fields and the same workflows. They’ll know how to use the tool as they’ve always have and be able to focus on getting to work that much quicker.

Secondly, It will help with reporting. With all the teams using the same workflows and fields, we have a consistent set of data across all teams with which to run reports and measure performance. While you should NEVER compare teams’ velocities to each other (as each team will value story points differently, therefore you will never get an oranges-to-oranges comparison), you can measure most other things consistently across the teams if they are all working from the same field set.

And the third, and I’d argue the most important reason to consider standardizing your schemes is your sanity. Look, there is only so much JIRA Admin to go around. If you are lucky, your organization has two or three of them to help out – but that’s usually for the largest organizations. Most small and medium businesses, it’s on you. Having standardize schemes stops you from having to play “What impact will this change have?”. You’ll know, and be able to get on with it that much faster.

But what about the exceptions?

Look. There will always be exceptions. A process or team that doesn’t adhere to normal business models. My Approvals workflow is such an example. But it’s your job to make it just that: An Exception. If you standardize your workflow, any change that is proposed by a team using the standard schemes should be measured on “How will this benefit the entire group?”. If it’s a benefit, it should be applied to the entire scheme. No breaking off a scheme to apply to a specific team. It’s all or none.

This, in and off itself, can help limit JIRA bloat. If a team has to defend why their change will benefit everyone, not just themselves, they will sometimes decide it’s not worth it.

So, where do I get started?

Honestly, I’d start with the screen schemes. And the first step is to talk with each and every one of your teams. Figure out what fields each one is using, and how they are using them. This will be time consuming – just a fair warning. But to do this successfully requires you to put in the work and really get to know your teams and how they work within JIRA.

Once you have that, sit down and compare your notes from all the teams. (You did take notes, right?) Find similarities, and try to get down to the fields you can get away with to allow as many teams to work as they already do as possible. Try to avoid fields that are used by only one team – opting instead to put that information in another field if possible, or the description. Take this time to use your test instance to make the changes. This will be both your testing and give you a way to demonstrate the change in a real way in our next step.

Now that you have your proposed standardized fields, this next step is probably the most important. Go back to each team, and present the new scheme proposal to them.

No, Seriously. Review what changes will impact their team specifically, and explain why you are making each change. Get feedback. Listen. Negotiate. Being willing to work with each team on this process is how you build buy-in, and it’s how you will make this process a success. Not everyone will be a 100% happy, and you may have to add some fields back in, but you can get them to a point where they can at least agree to the change. Again, I know this will take time, but having gone through this process myself, it is absolutely required.

So, you have con-census and buy in from your teams, and you have the changes ready to go in your test instance. You are now ready to push this to production..which in vanilla JIRA isn’t going to be fun. Without any plugins, you will need to manually recreate the new schemes in your production JIRA Instance, then assign all the appropriate projects to it. Or – you can use a tool like Botron’s Configuration Manager to copy over the new config as it exists from your test to production. Either way, I usually schedule a full change window for this during a relative down period for the instance (which invariably ends up being Saturday). No need having people try to work around you as you make these changes.

And that’s your Screen schemes standardized! When you are meeting with the teams, they may not understand why you are doing this. Stick with it – they will see the benefits eventually, and wonder why it wasn’t always like that.

But…uh…what about workflows?

Are you Forgetting something? Maybe you are,
Maybe you aren't - Are you Forgetting something? Maybe you are,
Maybe you aren't  Scumbag Brain

Yeah…workflows. I hadn’t forgotten about it – I was strategically putting it off. I promise!

My experience has been standardizing workflows is a bigger fight than screen schemes, which is why I chose to lead with that. People are willing to be flexible on fields – most people don’t use most custom fields anyways. But when you start messing with their workflows, you are messing with how they work. And yeah – that is part your job – but it’s going to be a fight. So start out with the screen schemes and build up some good will. That will make this easier.

With the Workflow schemes at least, you have the information at hand for each team. Teams are somewhat free-form in how they use fields, so you have to interview them first. But a workflow can only be used as designed. So start by looking at all the workflows for the projects you wish to standardize. Figure out the common points, and design a workflow (in your test instance!) that matches as many of them as possible.

Now that you have something designed, take it to those same teams. Explain to them what you hope to achieve, and demonstrate how the standard workflow you are proposing differs from the one that particular team is currently using. Go through the same thing with each team you did with the fields. Explain the benefits of everyone working on the same scheme. The object here is to build con-census from the ground up.

Look, I’m not going to sugarcoat it. People are going to disagree with you. Some are even going to do so passionately. Don’t make enemies of these people. If they care about how JIRA is being used THAT much, they are a person to have on your side. Figure out what their objections are, and figure out a way to address or otherwise work-around those objections. But show you are willing to work with their concerns, and they will be more willing to work with you.

Once you have all the concerns addressed and con-census built, schedule a change window and put it in place. You can do it manually, or by a plugin. Honestly, I keep harping on the plugin as an option as it eliminates some possibilities of you making a mistake. But if you are going to do it manually, at least export the workflow and re-import it into production, so you don’t have to worry about a mistake there.

And that’s it!

Look, this won’t be the easiest thing to do as a JIRA admin. People can be very passionate about how they work, and if you are seen as messing with that, they can get upset. But remember that they are upset because they do care, so if you can show that you are willing to listen and work with them, it should help.

So I know everything else in the world has been going a little crazy right now. I really hope you all survived the transition and ramping up your teams to work remotely. I’ve been stuck at home anyways while I recover from the surgery, but even before that I’ve been a remote employee for some time, so it still business as normal for me. I can write a post on tips and tricks from working from home, but it’s all likely advice you’ve heard elsewhere already, so unless requested I don’t have a plan to write on that topic. But if you are interested, let me know!

Don’t forget you can subscribe to receive new posts from this blog via email! To subscribe, just put your email in the form below this post! We also have an officially Unofficial Atlassian Discord chat. Swing in, ask questions, and see what’s going on! https://discord.gg/mXuRsVu

But until next time, this is Rodney, asking “Have you updated your JIRA issues today?”

Spring Cleaning for Custom Fields

Well, I’m back…or at least upright. Thank you all for the well wishes I’ve received. As much as I hate not posting an actual article, I really did need the week off to recover. But now it’s time to get back to it. As I’ve stated, this week’s article was going to be a reader selected topic. It seems there was some demand for each, but the clear winner was…

I’ll be tackling the other articles in the coming weeks, but lets look at Getting those custom fields under control this week.

A bit of a confession here.

First, let me say that as Admins, I think each one of us has various strengths and weaknesses. Me, for example. My obvious strength is Systems. I can tune almost any system to run smoothly. I can learn new back-end technologies with almost no lead-time and still successfully deploy.

However, my weakness as a JIRA Admin is I’ve never been able to execute a successful Cleanup. I’ve known the problem, and I’ve worked to fix it. But I’d rather be a con-census builder than a dictator when it comes to JIRA, and people LOVE their custom fields.

To manage this, I’ve always put safeguards to keep things getting worse. I re-use fields rather than create new ones, and really analyze whether new fields are needed. But on systems I’ve taken over, it’s rare I get a group to give up a field that was already existing.

That being said, if you want to learn from a real master of this topic, I’d highly recommend Rachel Wright’s JIRA Strategy Admin Workbook.

However, I’m going to give you tools to analyze the problem, see what fields aren’t being used, and give you the data to say to people “This…this is a problem we can fix.” Then I’ll show you how to go about cleaning things up. So, lets learn together!

Why does this even matter?

So, you might be asking yourself “JIRA is all about the custom fields, why should I worry about how many there are?” That, honestly, is a fair question. But the reason you should care is performance. It’s just a fact: The more custom fields your system has to parse through, the slower things like creating an issue, searching, etc will be.

Average response times per action (in milliseconds): custom fields test. Courtesy Atlassian.

Take a look at this chart above. This is from some testing Atlassian did. If we look at JQL Searching alone, at 1400 custom fields, the performance is abysmal at almost 3 seconds. However, doubling the custom field count to 2800 triples that number to nearly 9 seconds.

And yes, this is an extreme case. But users will notice a delay of half a second. You will get comments saying JIRA is slow even at that number, trust me. And it doesn’t take too terribly many custom fields to get searching to be that slow. That is why your custom field count matters.

Analysis

It is said the first step to fixing a problem is admitting you have a problem. Check. The second step it would seem is to figure out how big the problem is. Unfortunately, for this part we are going to have to go directly to the database.

For all my databases, I like to keep an administrative read-only account that will allow me to go in and query from time to time. This helps with making sure things are running properly, or diagnosing problems every now and again. Just make sure it’s read only so you don’t mess with something that aught not to be messed with.

Anyways, Atlassian has some queries that will help you figure out what fields are being used in your system. You can find all of them available here:

All the queries below I have confirmed are still working for MySQL 5.7 and JIRA 8.5.3.

Unused custom fields

select count(*), customfield.id, customfield.cfname, customfield.description
from customfield left join customfieldvalue on customfield.id = customfieldvalue.customfield
where customfieldvalue.stringvalue is null 
and customfieldvalue.numbervalue is null 
and customfieldvalue.textvalue is null
and customfieldvalue.datevalue is null
and customfield.CUSTOMFIELDTYPEKEY not like '%com.atlassian.servicedesk%'
and customfield.CUSTOMFIELDTYPEKEY not like '%com.pyxis.greenhopper%'
and customfield.CUSTOMFIELDTYPEKEY not like '%com.atlassian.bonfire%'
and customfield.CUSTOMFIELDTYPEKEY not like '%com.atlassian.jpo%'
group by customfield.id, customfield.cfname, customfield.description;

So, the first logical question is to ask is what fields are just not being used – period. These can probably be thrown out easily enough. Just a word of caution with this query: It can return fields that are being used by Apps using their own datastore. I’ve modified it to at least throw out results from JIRA Service Desk, JIRA Software, Capture, and Portfolio So make sure you are vetting this list for such false postitives. But if it’s a field you or your team has created, and it’s on this list, it might be time to have a discussion on whether it needs to be around.

Custom fields that have not been updated after date (YYYY-MM-DD)

select field.id, field.cfname from customfield field where field.cfname not in (
select item.field from changeitem item
JOIN changegroup cgroup ON item.groupid=cgroup.id
where item.fieldtype='custom' and cgroup.created > 'YYYY-MM-DD'
) and customfieldtypekey not like '%com.pyxis.greenhopper%'
and customfieldtypekey not like '%com.atlassian.servicedesk%'
and customfieldtypekey not like '%com.atlassian.bonfire%'
and customfieldtypekey not like '%com.atlassian.jpo%'

So this one returns fields that haven’t been used after a date. You’ll have to modify the query by replaying “YYYY-MM-DD” with a date you are interested in, but this one is handy for an end of year review, as it can answer “What fields have we not used this year”. If your teams haven’t touched the field in that long, you have a strong case for it’s need to go away.

Custom fields with low usage

select count(*), customfield.id, customfield.cfname, customfield.description
from customfield left join customfieldvalue on customfield.id = customfieldvalue.customfield
where customfield.CUSTOMFIELDTYPEKEY not like '%com.atlassian.servicedesk%'
and customfield.CUSTOMFIELDTYPEKEY not like '%com.pyxis.greenhopper%'
and customfield.CUSTOMFIELDTYPEKEY not like '%com.atlassian.bonfire%'
and customfield.CUSTOMFIELDTYPEKEY not like '%com.atlassian.jpo%'
and customfieldvalue.ID is not NULL
group by customfield.id having count(*) < 5
order by count(*) desc

Unlike the first query, this one will return fields that do not meet a certain threshold of usage. As written, this threshold is 5 issues, but you can adjust it by changing the number in “count(*) < 5”. As a rule, I like to keep this to one-tenth of one percent of the total number of issues in your instance.. So if you have 400,000 issues in your instance, the threshold would be 400. But that is a personal preference, you should set this wherever it makes sense for you.

So, what now?

You know have your information, and you know what fields are not being used. You can just go and delete them, right?

Well…

Not really. You should do some more digging. Who asked for that field to be added? What was their business case? Was that field just recently added? For those with a small usage, are they only being used in a specific project? If so, what is the usage within that project?

Also, use Botron Power Admin to see where all the field is being used (or not used). This might tell you who you need to talk to as well.

This should help you narrow down your candidates further. Once you do that, go back to whoever requested those fields. Take the data from your queries, and let them know that the fields aren’t being used, and are on a list to be removed. You can also include documents on how custom fields impact performance for everyone. If you are lucky, they’ll agree with the data and you can remove it. If not, start negotiating. See why they think they need a field they aren’t using. See if there is a way you can give them their ask without the field.

Custom Field optimizer (Data Center)

For those of you on Data Center, JIRA has a built-in Custom field optimizer you can use. It will go through and find fields that are not used often, or otherwise “misconfigured” (As defined by Atlassian).

As you can see, it’s alerting on a field in my instance that has a global context. This was one I specifically created to test the optimizer, so it’s Ideally, you should limit your custom fields to only be available to the projects that need it. So you can use this tool to find those with global contexts and if necessary, fix them.

Merging Fields (App Required)

So, lets say you have a duplicate field. How do you copy the information from one to another so you can delete one without losing data?

Well, as you can guess, this isn’t something that is able to be done in vanilla JIRA – so we have to turn to the Atlassian Marketplace – and specifically Adaptavist’s Scriptrunner plugin.

Now my groovy – it needs some work. But Scriptrunner has a number of built in scripts, including one to “Copy custom field values”. This does what it says on the tin. It takes the value in a source field, and places it into a destination field, overwriting what was there. For most cases, this won’t be a problem for reasons I’ll talk about later, but do keep this in mind.

To use this, you will first need to save a filter that will limit what issues it will copy the fields on. I use something along the lines of:

"Custom Field A" is not EMPTY

This will return all issues that has the custom field set. You can add another clause to weed out those that have the destination field already set, and handle combining those manually. In most cases, each of the duplicate fields would have only been used on a project or two a piece, and have no overlap, so this shouldn’t be too much of a concern, but as always do your research first and make sure that is not the case in your situation.

Now, I shouldn’t have to say this, but always test any changes you want to make before doing it in production. That’s very true here. This built-in script isn’t perfect, and you may have to pick it apart and modify it a bit to get to your use case. The only way to know if this is required is to do your testing.

That being said, I asked my colleagues at Coyote Creek about merging fields, and Neil Taylor gave me this as a solutions he’s used before. From testing it, it’s the easiest to implement and execute that I’ve found, and should handle most of your use cases.

So time to get cleaning!

So you know how to find fields that are good candidates to removal, and have an action plan to migrate the data should they have anything worth keeping in them. What are you waiting for? Get out there, start conversations, and get yourself on a better path to better JIRA Performance.

Don’t forget to subscribe to receive new posts by email. It’s the easiest way to be notified as soon as the posts go live. You can sign up with the form at the bottom of the post. And don’t forget to check out our Atlassian Discord Chat!It’s always interesting to see who has popped up there and what’s being discussed. https://discord.gg/mXuRsVu

But until next time, this is Rodney, asking “Have you updated your JIRA issues today?”