Tag: Jira Installs

Getting started with JIRA Data Center: Pt 2 – Updating JIRA to JDC

Rodney N.Leave a comment

So, when we left off as week, we had all the support system ready to go. As a review, we have the NFS Server, database, and Load balancer – each needed for JIRA Data Center to work to it’s fullest potential. Now we need to convert the actual JIRA instance into our first node for JIRA Data Center.

First, I want to address the elephant in the room. You may have noticed that this blog post is being released a little later than normal. That is because Atlassian released another vulnerability report today. This time its for Confluence Server and Data Center. I won’t be covering it today – I’d much rather get back to the JIRA Data Center install. But I’ll post it here for your own research

Also, as a cleanup note from last week, I forgot to mention one thing. All the systems you use should be configured to use the same NTP server for time synchronization. That is to say not the same *pool*, but the same *server*. What, me? Make that mistake? And only find it while writing this article? Never! </sarcasm>

Okay yeah, I did that. I am not perfect, and sometimes some things do slip by. But I don’t want you to make that same mistake, so I figured I’d own up to it now and not let you fall into the same problem.

Installing the JDC License

Before we do anything further, we need to install a JDC license key into JIRA. That way once we get things done and bring it online, you won’t have any issues.

To do this, we need to find our Server ID. This can be found by going to System -> System Support, then looking for “Server ID” on that page.

After you grab this, go to https://my.atlassian.com if you are getting a trial license. And honestly, I’d recommend you get a trial license while you are doing your initial testing. No need to put money down while you are doing a proof of concept.

Click on the “New Trial License”, then on the next screen select your JIRA flavor from the drop down (Either JIRA Software or JIRA Service Desk). Then once the rest pops up, select Data Center. You will then enter your Server ID we just grabbed.

This will generate a trial license that lasts 30 days. You can renew this a number of times, but it’s not infinite, so use this time wisely.

Copy your new trial license, then in your JIRA instance, go to Application->Versions and licenses, and paste it in. Then you are done with this step, and we can begin making JIRA into a proper node.

Moving the Shared Home.

From here on out, we are going to be following the doc “Installing JIRA Data Center”

Per the guide, we are setting up our shared. First, shutdown your JIRA instance.

systemctl stop jira

Once it has stopped, navigate to your JIRA Home Directory, and copy the following folders to your shared NFS directory.

  • data
  • plugins
  • logos
  • import
  • export
  • caches

Once copied, be sure to change the owner back to jira so that it can work once you bring the system back up.

chown -R jira:jira /data/jira/sharedhome

And this part is done! One step closer to having JDC up and running.

Setup your cluster.properties file

Now that we have the shared home setup, we need to setup the file that tells JIRA that a) It’s a JDC Node, and b) where to find the rest of it’s files. This is the cluster.properties folder, and it’s placed in the root of the local home folder.

This file will have the following settings:

# This ID must be unique across the cluster
jira.node.id = jnode1
# The location of the shared home directory for all Jira nodes
jira.shared.home = /data/jira/sharedhome

As a best practice, I like to put the system hostnames into the node id, that way we know where to look when we are having issues, but honestly it’s arbitrary, so use whatever works. The only requirement is that each node has a unique name.

There are some more options you have in the cluster.properties file, but unless you are having problems, you shouldn’t need to use them. But, I want you to at least be aware of them.

As a last setting, if you are running on linux (like we are), go to your <jira_install>/bin/setenv.sh file, and add the following line.

ulimit -n 16384

Now we need to unblock two new ports that JIRA will use to communicate between nodes.

firewall-cmd --permanent --zone=public --add-port=40001/tcp
firewall-cmd --permanent --zone=public --add-port=40011/tcp
firwall-cmd --reload

At this point we can start JIRA as a data center instance!

systemctl start jira

If all goes well, when JIRA will come up as a single node Data Center instance.

Standing up additional nodes.

Now, we can really take advantage of the super power of JDC: Adding nodes!

Now, if we want to go easy mode, we could just clone the VM and modify the cluster.properties file. But we are JIRA Guys, we don’t do easy mode here!

From a fresh install, we need to install the same version of JIRA that is on our first node. After this install, we need to see if we are using the same userid and group id for JIRA. If not, it will cause problems with NFS. So run the following on all nodes:

id jira

If any differs, it needs to be changed to match the existing nodes. Note this also means you will need to change the file permissions that belong to that particular JIRA user on that node. I could go through the guide on how to do this, but someone else has already done a much better job than I can.

With that minor annoyance out of the way, we can copy over the local home folder. The best way I found to do this is to tarball the entire thing up and scp it to the new node.

tar -czvf jira_home.tar.gz <jira_local_home>
scp ./jira_home.tar.gz <new_node>:~/

Once on the new node, unpack the tarball and move the home folder to the appropriate place. If you used default locations for both nodes, you can probably just unpack it at the root folder and it will find it’s way to the right place. Then we just make sure the permissions weren’t broken:

chown -R jira:jira <jira_local_home>

Next we modify the cluster.properties folder, changing the jira.node.id to something unique in the cluster. As an added note, if your nodes do not resolve to DNS, you will need to add the following to this file.

ehcache.listener.hostname = "<ip_addr_of_node>"

You will also need to do this for the existing nodes as well. I also like to take the extra step of adding the file share, database, load balancer, and all nodes to the hosts file of all systems involved. That way even if you do have a DNS Failure, your cluster won’t die. Host entry take the following format:

<ip_addr>    <fqdn>  <alternate_host_name>

An ounce of prevention is worth a pound of cure, especially when IT decides to move up an upgrade of a DNS node without telling anyone. Yes, that’s a true story!

At this point, you should make sure that the system has the shared home mounted. You can do this by copying the entry for the NFS share from the first node’s /etc/fstab file and placing it into the second. Then make sure the folder structure up to the mount point is present. You can uses the -p flag on mkdir to make all required folders in one command.

mkdir -p /data/jira/sharedhome

You can now mount the share with the mount command.

mount /data/jira/sharedhome

Take a second and test that the JIRA Account can write and see all files in there. Now unblock all the correct ports you need for JDC:

firewall-cmd --permanent --zone=public --add-port=8080/tcp
firewall-cmd --permanent --zone=public --add-port=40001/tcp
firewall-cmd --permanent --zone=public --add-port=40011/tcp
firwall-cmd --reload

If everything looks good, start up JIRA as you would on any other system, and follow the logs. IF all goes well, you should be able to access the node by going directly to the IP Address for it.

At this point, check the copyright info at the bottom. You should see the new node name at the bottom after the version info:

Log in, and check the Health Check, if all is good you should see green check marks on the Cluster.

Now the only thing to do is add it to your haproxy config, restart haproxy on your load balancer, and start sending traffic to your new node!

And you made it!

It’s an involved process, I tried not to lie about that. But it’s not an Everest sized challenge. You might have to get a bit outside your comfort zone and play with some technologies that as a JIRA Admin, you haven’t played with before. But my view on it is in order for us to grow as Admins, sometimes we need to step outside our comfort zones. But at least you have a road map to help guide you on your Data Center Migration!

I’m still looking to do a Lightning round Q&A session next week, so get your questions in. Anything from how I do testing and capture images for the blog, to how do you deal with a particular challenge, to what kind of server(s) do I run, it’s all on the table.

Also have an article I’m excited to write up for the New Years – which incidentally enough, is a blog day.

So until next time, my name is Rodney, asking “Have you updated your JIRA Issues today?”

Getting started with JIRA Data Center: Pt 1 – Support Systems

Rodney N.6 Comments

Okay, okay, you guys talked me into it. Today and next week we’ll discuss when you should start thinking about Data Center, what support systems you will need to put into place, and what the process looks like for converting your single-server instance into a multi-node JIRA Data Center instance.

It was actually an email from one of you that made me decide to cover JIRA Data Center. I love hearing from people about how much they are learning from this blog, So don’t be afraid to send me a comment, use the contact form on the blog, or send me a DM on LinkedIn.

Now, some forewarning here. A Data Center install is an involved process. This is something you absolutely should practice doing on a test instance. Actually no, you should practice it several times. It’s not something you want to just do on production. So, without any more delays, lets get to this.

When should you look to convert to Data Center?

So, question time. When should you look to migrate from JIRA Server to Data Center? Is this something that will even be worth the expense and time?

This is a question that I’ve actually struggled with too. I actually pushed back against a Data Center Migration for a while there, with my reasoning being that we weren’t experiencing any significant problems with performance, why take on the extra cost and effort.

It was actually a mid-day downtime event that made me reconsider. The VM host that JIRA lived one unexpectedly went down. Now the VM infrastructure wasn’t my responsibility, but the resulting corruption of the JIRA Index was. JIRA was actually down for an additional 1.5 hours because we had to rebuild the index from scratch.

So I’m going to tell you now, don’t be me. Use actual numbers and metrics to inform your decision. Atlassian recommends you look at three things. These are not a hard and fast checklist, but a way to start the conversation about whether this is right for your organization.

1. Active User Count

The first Criteria is User base. Atlassian’s own studies have shown that organizations typically start running into performance issues on JIRA Server when supporting between 500 and 1000 active users. So if your instance has a peak load of around 450 users – it might be time to bring this up.

2. Performance Degradation

The second is actual performance. If you are experiencing regular performance degradations after you’ve done every optimization you can find – it might be time to bring this up. You can only “grease the track” so much before your single node can’t support any more.

3. Downtime and Outages

The third factor to consider is how critical your instance or and how tolerant you can be of downtime and outages. If your business needs dictate that JIRA has to be up, period, you guess it. It might be time to bring this up.

Now these do provide some numbers and guidelines, but they leave a lot up to your judgement. Take your time and consider these things carefully. This is not something to shout “Leeroy Jenkins” and run into head-first.

What support systems will I need to setup?

So you’ve looked at everything and decided, “Yes, Data Center is for me.” What next? Well, that’s actually going to be the focus of rest of today’s post. In order for JIRA Data Center to work, each “JIRA Server” Node needs access to a common shared set of resources. You can actually read more about it in this week’s Document, simply titled “JIRA Data Center”

From Atlassian JDC Documentation

As you can see, the three things we’ll need are a shared Filed System, a Load Balancer, and a Shared Database. As far as Databases go, JDC supports the same selection as JIRA Server. This means we can setup the Database the same way we did for Server, only we’ll need to tweak the settings a bit to make it friendly for Network use.

For the file share, I’ll be setting up a dedicated NFS Share Server for this function. I’ll also be setting up HA Proxy for the Load Balancer. Both of these, as well as the Database, will be running on CentOS 7 systems. These are both my preferences…you could use SMB/Windows File shares if you were running in a windows environment. Or you can run Nginx as your load balancer. If you have the funds, you can even run an F5. As I’ve stressed multiple times: Follow the supported platforms sheet, but use what you or your team knows.

Assumptions

Alright, so I am assuming a few things here. First, I am going to assume that you are at least familiar with how I setup JIRA Server, based on my posts here, here, and here, and that you have set up your Server instance following those instructions. I am also assuming that the database is currently on the JIRA Server.

These are assumptions I need to make in order to write this guide, as I need to know where you are starting from. However, I also think that you are smart enough that where your configuration is different, you can figure it out from what I’ve provided.

Setting up an NFS Share for the Shared Home

So…this is annoying. There doesn’t appear to be a good guide from Atlassian in setting this up. But that’s okay, I’ve setup NFS before for other purposes, so we got this.

Start with a fresh CentOS 7 machine. Our first job will be to install the package nfs-utils.

yum install nfs-utils -y

After this we’ll make a directory where the share will live. For monitoring ease, I’m also going to put this on it’s own drive separate from the root filesystem, but one step at a time.

mkdir /var/jdc-share

Now that we have a directory for the share, lets map it to a drive. To do this in a sustainable way, we need to find the UUID of the new drive – which for my example is /dev/sdb1

blkid /dev/sdb1

We then take this, and using the text editor of your choice, modify /etc/fstab, adding the following line:

UUID=e64ff994-a046-407e-96fe-3bc8ba149254 /var/jdc-share xfs defaults 0 0

If you have your fstab correct, all you should have to do is mount the folder. I will also check df -h to be sure it mounted as I expected:

mount /var/jdc-share
df -h

Next we need to make sure the permissions are correct so that we don’t have any issues when we configure it for NFS. To do this, we need to set the folder’s permissions and file ownership:

chmod -R 755 /var/jdc-share
chown nfsnobody:nfsnobody /var/jdc-share

Now that we have the filesystem prepared, we can configure the NFS service to actually share that folder. Open up the /etc/exports folder in the text editor of your choice and add the following line

/var/jdc-share <ip range of JIRA nodes>(rw,sync,no_root_squash,no_all_squash)

For the IP range – you might need a network engineer to help you. Most of the time though, you can get a close approximation by doing the following. Take the IP address of your first node or JIRA Server, as appropriate, and replace the last octect (number) with 0, then add a /24 to the end. So if your JIRA Server’s IP is 172.16.1.63, your IP range will likely be 172.16.1.0/24. This only works if all your JIRA nodes will be in the 172.16.1.0 subnet! Talk to your Network Engineers or whoever provisions your IP addresses to confirm!

So with that configured, start the following services:

systemctl start rpcbind
systemctl start nfs-server
systemctl start nfs-lock
systemctl start nfs-idmap

This will start your services. Now we need to make sure the Firewall won’t block your incoming nfs requests:

firewall-cmd --permanent --zone=public --add-service=nfs
firewall-cmd --permanent --zone=public --add-service=mountd
firewall-cmd --permanent --zone=public --add-service=rpc-bind
firewall-cmd --reload

Now it’s time to test. Remember, you are not done until you’ve confirmed it’s working yourself. Go to your JIRA Server and install nfs-utils, the same as we did for the NFS Server. Then to a temporary mount using the following command:

mount -t nfs <ip address of nfs Server>:/var/jdc-share /mnt

You can use a domain name for this, but DNS then become yet another point of failure, so for the support systems, I like to use IP addresses where possible. If all goes well, you should see no error. Try writing a file or two to /mnt/ from the JIRA Server, and see if you can see it on the NFS Server.

If all looks good, we can go ahead and enable those services for the NFS Server, and add a fstab entry to all your future JIRA nodes, then mount the share.

On NFS Server:
systemctl enable rpcbind
systemctl enable nfs-server
systemctl enable nfs-lock
systemctl enable nfs-idmap

This is the following line that needs to be added to the /etc/fstab in every JIRA node in your JDC deployment:

<ip address of NFS Server>:/var/jdc-share /data/jira/sharedhome nfs defaults 0 0

Make sure the mountpoint ‘/data/jira/sharedhome’ is created on each node, then manually mount the share to that node. By adding it to the /etc/fstab, it will also auto-mount on each boot.

umount /mnt
mkdir -p /data/jira/sharedhome
mount /data/jira/sharedhome
A bit of a different order, but still works!

And that’s the Share. We can’t really use it until we go to transform our JIRA Server into JIRA Datacenter – unlike the Load Balancer and Database. But we’ve at least tested it working as expected.

Setting up HAProxy for use with JIRA DC

Unlike with the file share, Atlassian has some documentation around setting up your Load Balancer.

So, for this we’ll also be starting with a fresh CentOS 7 instance. To install HAProxy, we’ll run the following command.

yum install haproxy -y

This will install the application on your system. To configure it, first take a backup of the file /etc/haproxy/haproxy.cfg, then open it in the text editor of your choice. 

cp /etc/haproxy/haproxy.cfg /etc/haproxy/haproxy.cfg.example
nano /etc/haproxy/haproxy.cfg

Once open, remove the following sections from the default config:

  • frontend main
  • backend static
  • backend app

After this, add the following to the bottom:

#Begin JIRA Configuration
frontend ft_web
  bind 0.0.0.0:8080
  default_backend bk_web
  
backend bk_web
  balance roundrobin
  cookie JSESSIONID prefix nocache
  server s1 <JIRA Server IP>:<JIRA Server Port> check cookie s1
#End JIRA Configuration

Once you enter this, start haproxy with the following command:

systemctl start haproxy

You should be able to go to port 8080 on the Load balancer server and get to your JIRA instance. This assumes that port 8080 is open on your JIRA server and the load balancer. If it passes, enable the service so that it survives a system restart:

systemctl enable haproxy

A note here: My configuration assumes you are going to put a proxy in front of the load balancer to handle SSL translation. You would do this similar to how we did it for JIRA Server, just pointing to the load balancer instead of the JIRA application. You can also set up HAProxy to handle SSL directly as well, but lets keep thing easy here.

Database?

The database is the easiest part of all this. If you’ve already setup the database as a remote resource, congrats, you are already done with this section. However, if you haven’t, please read on.

From yet another fresh CentOS install, follow the database setup we went through with JIRA Server. All the settings will be the same for this, save for a few details. First, we need to tweak the SQL statement that grants access to the JIRA User.

By default it looks like:

GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,DROP,REFERENCES,ALTER,INDEX on <JIRADB>.* TO '<USERNAME>'@'<JIRA_SERVER_HOSTNAME>' IDENTIFIED BY '<PASSWORD>';

However, we need to make sure JIRA can log in remotely, so instead of “<jira_server_hostname>”, we’ll be putting ‘%’, so that it now looks like:

GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,DROP,REFERENCES,ALTER,INDEX on <JIRADB>.* TO '<USERNAME>'@'%' IDENTIFIED BY '<PASSWORD>';

Granted, you could add a grant for every JIRA hostname, but in reality this adds alot of overhead for only marginal security gains, so it’s not really necessary.

Next we need to add a firewall rule to allow traffic to mysql:

firewall-cmd --permanent --zone=public --add-service=mysql
firewall-cmd --reload

And that will be your Database ready to host a JIRA database. Now we just need to migrate a JIRA DB to it. First thing you should do is shut down your JIRA instance as to guarentee there will be no changes to the data while you work. If you setup your JIRA instance as a service, enter the following.

systemctl stop jira

Now go to your current JIRA host, and enter the following command, using the JIRA DB’s username and password for that host:

mysqldump -h localhost --user=<JIRA_DB_USERNAME> --password=<JIRA_DB_PASSWORD> --single-transaction --quick --opt <JIRA_DB> | gzip > "jiradb-$(date +%Y-%m-%d).gz"

Transfer the resulting file, which should be named “jiradb-<today’s date>.gz”, to your new centralized DB server. Once it’s there, import it into that database using:

gunzip < jiradb-<today's date>.gz | mysql -u root -p <JIRA_DB>

You should take a moment here to connect to the new database with a remote tool, like mysql workbench, using the JIRA DB username and password. Make sure you can see your JIRA Database as you’d expect it before we can move on. If you can connect to it, JIRA should be able to connect to it, assuming no network obstructions.

Once you completed that, you should be ready to cut JIRA over to the new database. Go to the JIRA home folder on your JIRA Node. There you will find a file called, “dbconfig.xml”. Take a backup of it, then open that up in your text editor of choice.

In here we are looking for the fields “username”, “password”, and URL. Change your username and password to match your new database system.

Now startup JIRA, and monitor the logs to make sure it connects cleanly. Also, once it’s done loading, go into the UI to make sure everything looks normal. Also check under the System -> Troubleshooting and Support tools that there are no errors, and check the System -> System Info to make sure it’s connected as you expected.

If everything looks good, congratulations! Your system is now ready to be converted to JIRA Data Center – which we’ll focus on next week.

A quick note here: This only works if you are staying on the same database platform. If you are using this opportunity to migrate to another Database platform, like moving from MySQL to PostgreSQL, I suggest you read the following Documentation. However, the gist of it is, you will need to run an export of your entire instance, stand up a new instance on the new Database Platform, then Import your backup into that new instance, followed by copying over the attachments.

Can’t I have one system do all three support roles?

In an ideal world, yes. Save the resouces, run only one VM. We don’t live in an ideal world. The idea here is to spread the risk around to multiple systems so that any one point is less likely to be a problem. Having one machine perform multiple roles increases the complexity of that system, and therefore increases the likelihood of something going wrong. As my engineering professors used to say: “Keep it simple”

But you have some pretty big single points of failure you got right there.

This is a valid criticism of my configuration here. With enough time and resources, ideally you’d want to make every system here redundant – and they all support redundancy. However, is it always worth it? My goal is to point you in the right direction. In my lab setup, I have one VM server, with limited resources. There is also time to consider. I have a self-imposed deadline for these articles as well.

But if you have the time and resources, you should definitely research how to run each of these services redundantly. If our goal is no downtime, it will do a lot to guarantee that. Understand this is an example project, and not a production system.

So what’s Next?

Well, next week we’ll talk about converting your JIRA Server system into a JIRA Data Center Node, and then what you need to do to setup each additional JDC Node after that.

Also a note about the coming weeks. We are about to enter the holiday season here in the United States, so I’ll actually be off work (though on call) for the blog posts on the 25th and the 1st. Given that, I’d like to do something special for those. So, send me your quick JIRA questions and I’ll do a lightning round Q&A, assuming I get enough questions in. And remember, I am always willing to take on reader requests for topics, so even if you don’t think your question is small enough for a lightning round, ask it anyways! This very post started as a reader request!

So until then, this is Rodney, asking “Have you updated your JIRA Issues today?”

Installing JIRA from scratch, Part 3: Reverse Proxies and Final Tweaks

Rodney N.1 Comment

So welcome back to the third (and final) post about installing JIRA from a clean linux setup. Last week we deviated a bit from this series, but there was a reason. I needed the extra time to study, and well, this is going to be a long post.

Study for what you say? I’m glad you asked! As of last Thursday, I can officially add this shiny thing to my resume:

So shiny…

I also took the opportunity to get my Scaling JIRA badge (ASB-SCJ) to extend my ACP-JA. But I did pass the test! Needless to say, this is an actual video me last week.

But with self-congratulations out of the way, I think it’s time we continue on with the series!

What were we working with again?

So, as it’s been a few weeks, lets take a look at where we are. We started with a clean CentOS install on a VM with the following specs:

  • OS: CentOS 7 Linux
  • RAM: 4 GB
  • Processors: 2
  • Root Hard drive of 30 GB
  • Extra Hard drive of 50 GB to house the database and JIRA’s home directory

On top of this, we installed MySQL 5.7 and JIRA Software 8.1.3, and had everything in a working state, but running off of port 8080.

However…Security Updates…

Fun fact, did you know that before I started with my current job, I had about three weeks of posts saved up to allow me to focus on my career for a bit? That means I had the Part 2 for this series finished and in the can about two weeks before it was published. And honestly, I had done the install and captured the pictures weeks before that. Which leads me to the reason I bring this up…

On Sept. 18, I got a nice message from Atlassian, letting me know about a security vulnerability. As the version I installed was not covered under this or any previous alert, I opted not to change my post to address it – opting instead to disclose the fact now. However, I do *highly* recommend you check back with this page regularly, and plan your upgrades and installs accordingly.

So why do we need a reverse proxy?

So, for most seasoned people, this will be obvious. But I’m going to cover it because it’s always helpful to explain the why of what we are doing. Especially when it seems like an extra step.

When Tim Berners-Lee created the original http protocol, he did not imagine a world where we’d be handling everything through websites. Honestly, he was just making a mechanism to share public research papers. Being public, he didn’t foresee the need for security.

The end goal of all this is to say, if you aren’t specifically using https (<-note the “S” at the end), every time you or someone logs into your JIRA instance, you are sending that password in clear text to the server. Sounds bad? It’s because it is.

The HTTPS protocol is how we fix this. It takes the web traffic you send, encrypts it in such a way that only the server can decrypt it, and sends it on to the server.

JIRA’s tomcat can do this encryption/decryption with some work and time – however, it’s not great at it. Combine that with the fact that it will use resources in your JVM heap that could go to JIRA, it’s generally agreed that this isn’t the best solution.

So we setup what’s called a “Reverse Proxy”. It sits in front of the web traffic, and handles the encryption/decryption much better than JIRA can. That is to say, JIRA will feel much more responsive while still staying secure.

Sounds Great, how do we get started?

First we need to ask: Which service we are using as a proxy? There are several options, with two being the ones most people choose: Nginx and Apache. Today I am going with Apache for two reasons.

The first, and likely most important for me, it’s what I know. I’ve been setting up some variation of LAMP servers for over 15 years now. It’s a system I know well by now.

Second, and likely most important for you: It’s the one I found documentation for on Atlassian’s knowledge base. When in doubt, follow the docs.

So…more reading material then?

You got it! In today’s reading corner, we have “Securing your Atlassian applications with Apache using SSL” from Atlassian. Thankfully it’s the only one we’ll need this week.

I’ll be deviating from it in a place or two (mainly in the Apache Configuration), but I will note where I do this. These changes are quality of life improvements, which makes my (and hopefully your) life easier.

SSL Certificates

If you are lucky, you will have someone at your company who can grant you access to – or otherwise get – an SSL Certificate for you. Unfortunately, you are the JIRA Administrator, so if you don’t have this resource, you will need to get one yourself.

You can get them from a variety of sources, but I recommend you stick with reputable Certificate Authorities. It would very much ruin your day if you went cheap just to find Mozilla or Google pulled that CA, causing your site to stop working across the internet.

For my home environment, I use a wildcard certificate from Comodo. No matter who you use, if they are reputable, they will have a knowledge base to help you through the process, so I won’t cover that at this time, instead assuming we are starting after you have received your certificate.

When I ordered mine last, I got it within the hour, so if you have to wait to get one, you won’t have long to wait.

So, what will I get when I finish

When you finish, you’ll likely get a zip or other compressed file containing, your certificate, several other certificates, and a key file. Save the key, it will look something like this.

Now you will notice my key file is missing. Please, for all that is good, keep your key secure at all times. With that file, anyone can decrypt the packets heading to your server.

Of what you can see here, we have three chain certificates, plus my certificate (Star_folden-nissen_com.crt). You will need to know what parts are what. Transfer all these files plus your key to your server’s filesystem, taking note where each is.

So, we’re finally going to do something on the server?

Yes! First thing first, we want to make sure Apache is installed on your system. Here we’ll use a variant on CentOS called httpd. This still very much is Apache, just with focus on only putting out the most important updates – in such a way that doesn’t break your configuration. 

yum install httpd mod_ssl

If you need to install it, you’ll see the familiar confirmation. Type “y”, followed by enter to proceed.

Looking at the configuration directory, we should see the following files

cd /etc/httpd/;ls -lah

For the next section, we’ll be focusing in on the conf.d folder.

Configuring Apache

I told you when I’d be deviating from the Atlassian Documentation, and where it is. I like to separate each service a reverse proxy is supporting into it’s own conf file, as it makes everything easy to look through. While not expressly explained in the documentation, it doesn’t go against it, but I’d like to be clear about any deviations I make and why.

Moving into the conf.d folder, we’ll see the following files:

  • autoindex.conf
  • README
  • ssl.conf
  • userdir.conf
  • welcome.conf

Really, the only one we’ll care about today is ssl.conf, however we’ll want to disable the rest from apache. This can be done by renaming the extension of each from .conf to anything else. For example:

mv autoindex.conf autoindex.bck

After you run this for everything but ssl.conf, you should see this:

Now lets dive into ssl.conf. Open it up with the text editor of your choice (I prefer vim, but nano is admittedly much easier to learn). You will see a bunch of settings required for ssl. We’ll keep most of these, instead commenting out anything between <VirtualHost _default_:443> and </VirtualHost> (including these two lines).

After this, we are going to create a new file, “jira.conf”, and entire the following.

<VirtualHost *:80>
    ServerName <subdomain>.<domain>.com
    Redirect Permanent / https://<subdomain>.<domain>.com/
</VirtualHost>


<VirtualHost *:443>
    ServerName <subdomain>.<domain>.com

    ProxyRequests Off

    <Proxy *>
         Require all granted
    </Proxy>

    ProxyPass / http://<subdomain>.<domain>.com:<port>/
    ProxyPassReverse / http://<subdomain>.<domain>.com:<port>/

    SSLEngine On
    SSLCertificateFile /path/to/your/cert.pem
    SSLCertificateKeyFile /path/to/your/privkey.pem
    SSLCertificateChainFile /path/to/your/first_chain.pem
    SSLCertificateChainFile /path/to/your/second_chain.pem
    ...
    SSLCertificateChainFile /path/to/your/nth_chain.pem

</VirtualHost>

A few notes here. You need to put in as many SSLCertificateChainFile entries as you have chain files, but be sure to include them all. This example list the files in a .pem format, but in reality they can be any number of different file extensions, so don’t think you need to chain it to be a .pem specifically.

But what if I have my JIRA on a sub-directory?

This is supported, but we’ll need to make Apache aware of this fact. Assuming your sub-directory is <subdomain>.<domain>/<contextpath>, we’ll get

<VirtualHost *:80>
    ServerName <subdomain>.<domain>.com
    Redirect Permanent /<contextpath> https://<subdomain>.<domain>.com/<contextpath>
</VirtualHost>


<VirtualHost *:443>
    ServerName <subdomain>.<domain>.com

    ProxyRequests Off

    <Proxy *>
         Require all granted
    </Proxy>

    ProxyPass /<contextpath> http://<subdomain>.<domain>.com:<port>/<contextpath>
    ProxyPassReverse /<contextpath> http://<subdomain>.<domain>.com:<port>/<contextpath>

    SSLEngine On
    SSLCertificateFile /path/to/your/cert.pem
    SSLCertificateKeyFile /path/to/your/privkey.pem
    SSLCertificateChainFile /path/to/your/first_chain.pem
    SSLCertificateChainFile /path/to/your/second_chain.pem
    ...
    SSLCertificateChainFile /path/to/your/nth_chain.pem

</VirtualHost>

Please note the subdirectory here includes a leading slash, but not a trailing one. Be sure yours looks like that as well, otherwise you will have problems.

Also keep note of what you have as <contextpath> – we’ll need it again here in a bit.

So what now?

Before bringing Apache online, we’ll need to test the configuration.

apachectl configtest

Assuming everything is alright, you’ll see a “Syntax OK”. It won’t catch every problem, but it will catch some, which is better than nothing.

Next we’ll need to allow firewall traffic to ports 80 and 443:

firewall-cmd --zone=public --permanent --add-service=http
firewall-cmd --zone=public --permanent --add-service=https
firewall-cmd --reload;

And finally, we need to tell SELinux that Apache can be allowed to do an HTTP Relay.

setsebool httpd_can_network_relay on

After all of these steps are completed, we can start, enable and test our JIRA proxy

systemctl start httpd
systemctl status httpd
systemctl enable httpd

And we’re done?

If you’ve done everything correctly, browsing to your JIRA Server on port 80 should automatically redirect you to port 443 and show JIRA.

Note: if you are using a subdirectory, JIRA will not come up at all. But you should still see a secure connection. We’ll fix this in a later section.

Notice the notification bar across the bottom. This is because we need to make JIRA aware it is running through a proxy.

On the filesystem, navigate to <jira_install_dir>/conf/, and open up server.xml in your text editor of choice. You should see something like this.

Under the connections section, add the following without modifying anything else in there.

scheme="https" proxyName="<subdomain>.<domain>.com" proxyPort="443"

When done, the connector should look something like this:

Remember that context path from earlier?

If you are hosting your JIRA on a subdirectory, you will need to make JIRA aware of that here as well. Go down to the context section, and where it says path, enter “/<contextpath>”. As with earlier, note the leading slash, and do not put a trailing one. It should look something like this when you are done.

Anytime you make changes to server.xml, they will not be reflected until you restart JIRA.

systemctl restart jira;tail -f <jira_home_dir>/atlassian/application-data/jira/log/atlassian-jira.log

This should restart JIRA if you have installed it as a service, and then bring up the log file for you to follow as it boots. Anytime you make changes to configuration files, it’s always a good idea to monitor the logs during startup – just to make sure the system comes up cleanly.

SO ARE WE DONE NOW!?!?!?!?!?

Uh…no. We have two last things to do. If we go to the site now, we no longer see that notification bar. HOWEVER, if we log into our admin account, we see the following message.

Here, we simply click “Update Jira’s base URL” and it should update to the URL you are accessing it from.

While we here, we also want to run a health check by going to Administration -> System -> Troubleshooting and support tools. If all the steps were followed, everything should be green. If not, the items not in a passing state will link to documents to help you fix it.

And finally, we want to remove the firewall exception we setup to initially access JIRA during setup.

 firewall-cmd --zone=public --remove-port=8080/tcp

And now, we are done!

Congratulations! You know have a production ready JIRA instance. It’s a good bit of work, but we’ve made it! Go, take a break, relax for a few minutes. There are still steps you will need to do like add users (or preferably, a user directory), setup projects, and in general be *that* guy at your company. But for the moment, this task at least is done.

It’s been quiet the journey. I get it, this isn’t a trivial process. To help, I’ve long had a checklist that I follow to make sure I hit each step – and I’ve decided to make it public. Hey, if it’s good enough for a pilot preparing for takeoff, why not use it as an Admin preparing for a Production instance. You can access it in the link below

And Some closing notes

I’ve seen some people reminding me that the Atlassian JIRA Cloud offering is still a thing, and requires a lot less setup. And you are right! There are customers who need will need something quick, and have no need to learn how to set it up the way I have. For these customers, absolutely use JIRA Cloud. It does have it’s limit, but for small shops, it’s really the best option.

That being said, there are companies that don’t have that option. Either internal policies or relevant laws say they cannot have a cloud instance of JIRA. And they may not know the first thing about setting up a production ready instance. Those are the people I wrote this guide for.

Knowledge isn’t meant to be trapped and kept forever. It’s for us to share after we learn it, so that it can live on well beyond our time. I consider this just the start of giving back the knowledge I have gained from the greater Atlassian community.

So – as of this writing, I’m not actually sure what next weeks post will be about. I have several ideas, just not sure which one to chose. If you have anything you would like to hear me talk about, please let me know in the comments below! Until then, this is Rodney, asking “Have you updated your JIRA issues today?”

Operation JIRaspi

Rodney N.Leave a comment

So, I was looking at the specs for the new Raspberry Pi 4’s when they came out in July, and something hit me. They now offer a version with 4 GB of ram. If you’ve followed me for a bit, you know this is my default sizing for a new self-contained JIRA instance. I’ve seen plenty of tutorials on how to setup MySQL and Apache, but maybe….I had an idea.

The problem previously with running JIRA on a Raspberry Pi before has always come down to memory. Even if you offload the database and proxy to another system, The Raspberry Pi’s previous record of 1 GB of memory just wasn’t enough. In the best case you’d get the heap thrown into swap space (not a place JIRA wants to be), and your service will slow to a Crawl. Worst case, the system would become so resource starved it would outright kill the JVM.

However, with the four gigabyte option now available, this might just work to run JIRA. The idea wouldn’t let me go. Considering my job loss at the end of July, I couldn’t justify the expense to actually test it. But now…

Why hello there!

No, but seriously, why?

Let me be clear, this experiment is made entirely out of impracticality. I don’t intend this to be my actual JIRA System at home. Even if I got this to run, there is no way that Atlassian would support it. This is entirely for exprimentation’s sake.

I will come again and conquer you because as a mountain you can’t grow, but as a human, I can”

Sir Edmund Hillary

I am doing this so I can learn. I want to see what JIRA does under such conditions. I want to challenge myself to find workarounds and fixes to problems as they come up. In short, I want to grow.

Challenges going into it

Given I know both the architecture of Raspian and JIRA, I can anticipate a few challenges already.

The first problem I will face is the Raspberry Pi uses an ARM based processor. However, Both Atlassian’s installer and the Java that is packaged with JIRA are both compiled against the x86_64 architecture. This means I know two things off the bat:

  1. I will have to do a tar.gz install. I’ve done one before, but it was more in the context of learning enough that I could automate upgrades with Ansible. I’ll have to rely on Atlassian’s documentation for this one.
  2. I will need to figure out what version of Java JIRA is shipping with, and install a version of OpenJDK to match it. You can force JIRA to use a different JAVA install by setting the appropriate setting in the <jira_install_dir>/bin/setenv.sh, so I’m not terribly concerned about this either.

That being said, I’d be surprised if these two are my only challenges. That being said, the only way to discover them is to go through with it, so might as well get started!

Documents List

As always, I’m including a list of documents I’ll be referencing. As always, I’ll look at the Supported Platforms document. While this experiment is most definitely not a supported platform, the document will yield clues to help me proceed.

Next I’ll be referencing the guide to doing an install from the archive. As stated, this is a process I’m not terribly familiar with, so it will definitely help.

As I’m already so far out of the Supported Platforms nest, I’m going to also use MariaDB to simplify DB installation. That being said, I intend to configure it as I would MySQL – to the extent I can. This will be another “I’ll discover the problems when I get there” deal.

Raspberry Pi 4 Specs

Looking at the relevant Specs for the Raspi4, we get:

  • Broadcom BCM2711, Quad core Cortex-A72 (ARM v8) 64-bit SoC @ 1.5GHz
  • 4GB LPDDR4-3200 SDRAM
  • Gigabit Ethernet

This is good news…mostly. A quad core at 1.5 GHz should be more than enough to run JIRA, and a 64 bit processor will allow us to run it without tweaking some setting down.

This processor and bus actually support PCIe, so we actually get a true Gigabit port on this raspberry pi, which certainly won’t hurt things.

Database Install and Setup

I’ve previously done a write-up on how to install a Database for JIRA, so I won’t bore you with that retelling. The major difference is Raspbian uses apt instead of yum, but the syntax and install method isn’t terribly different.

This part actually goes smooth. The only difference in process is that MariaDB didn’t generate a temporary root password, so I was able to run the mysql_secure_installation right away with no password. Needless to say, the password situation didn’t stay that way for long.

Preparing the Operating Theater

So I want to take a couple of steps to prepare things ahead of installing JIRA. The first thing I want to do is make sure I have OpenJDK 8 installed. I do this by running the following command:

apt-get install openjdk-8-jdk-headless

After the installer finishes, I double check it by running the following command:

java -version
Expected Output of java -version

I can see here that it did install properly. After we complete this, we’ll need to prepare a user account for JIRA to run under. You do not want your JIRA running as root under any circumstances – even in this experiment. 

useradd --create-home --comment "Account for running Jira Software" --shell /bin/bash jira

JIRA Install and Setup

For the most part I follow the document I listed earlier step for step. Test seemed to work really well and JIRA came right up when I tested it. Honestly, as I stated earlier I was expecting to have to modify the JAVA_HOME variable in <jira_install_dir>/bin/setenv.sh file, but that wasn’t even needed. It’s nice to be surprised when stuff just works.

The guide isn’t clear on where to setup the install and home directories, so I stuck to the defaults:

  • Install: /opt/atlassian/jira
  • Home: /var/atlassian/applicaiton-data/jira

So, time to add the JDBC connector, start it up for good, and get to configuring.

Finally a Problem

So at this point, I continued with the web based setup. Honestly, I couldn’t believe it was going so smoothly. The database connected fine, and from the log scroll I could tell it had setup fine as well. It was after the database setup that I ran into my first problem. My web browser just went blank. I attempted a JIRA restart, and same issue. I isolated it down the following error in the logs:

java.awt.AWTError: Assistive Technology not found: org.GNOME.Accessibility.AtkWrapper

Yeah….after a bit of googling, I found it was a configuration problem with openjdk. For more details, read this forum post:

The long and short of it is I needed to go into the file

  • /etc/java-8-openjdk/accessibility.properties

And comment out the line assistive_technologies=org.GNOME.Accessibility.AtkWrapper by putting a pound sign/hash/hashtag in front of it.

After doing this and restarting JIRA (again…), everything in the logs seemed to proceed normally. After waiting for a few moments for things to proceed, I refreshed my browser and it was back to the races!

Install Complete

I was able to complete the next few steps easily, and didn’t see another problem until I went to setup a sample project.

Hello Darkness, my old friend

Looking this one up, I found this article from Atlassian:

However, this ended up being a dead end as the mentioned setting was unset. Only thing I can guess at this point is it was a weird OpenJDK 8 bug. If I’m being honest, I always prefer Oracle’s JDK to OpenJDK for this reason. For Atlassian apps, oracle’s offering just seems to run better. However, a restart of JIRA and things completed as if it never happened, so I’m at a loss. I hate it when things just fix themselves.

And it’s over!

Everything is looking good and looks like we are ready to go! Looking at the system stats, I’m not seeing any evidence of it being overloaded. In fact it looks quiet conformatble:

I do notice some slowness on JIRA startup that I don’t notice on other JIRA systems. However, I’m so far out in the weeds here that it could be caused by any number of things, honestly.

The Systems health checks also passes with flying colors, So that’s also a plus!

So….what was this even?

I’m considering this a success. Starting this project, I wasn’t entirely sure it was possible, but all told it was. This could probably support a small team of 5 or 6 for some time before it needs to be upgraded to a server of some sort.

That being said, I wholly do not recommend anyone else do this in a production system. It’s definitely a curiosity, and cool, but far from practical. My main concern here is that the main system storage is an SD card – a thing that is not known to be entirely stable. You could supplement it with some USB drives in a software RAID, but even that feels risky. If you are looking to do something like this, I’d recommend just getting a traditional desktop and re-purposing it. You’ll come out more stable in the long run. But this was definitely a fun experiment.

Next week I’ll return to the “How to install JIRA” series with the finale, where we’ll take a look at some tweaks you need to make, as well as setting up an SSL reverse proxy. So until next time, I’m Rodney, asking “Have you updated your JIRA issues today?”

Installing JIRA from scratch, Part 2: JIRA

Rodney N.1 Comment

So, last week we took a look at the process to install MySQL to a server, and set it up to work with JIRA. I’d like to keep the momentum up with the actual JIRA install. Although this will feel like a lot of steps, I can usually do a DB and JIRA install in about an hour. It’s exhausting, but possible.

This isn’t a race though. Unless I have reason to rush, I’ll still take my time, double check each step, and use a checklist to make sure I’m not missing any steps. An ounce of prevention is worth a pound of cure, after all. So, lets get started!

Planning

The first question you should have in mind is your target version. Technically, you should have considered this first as it will impact what Databases and platforms are compatible, but I had already decided that I would be targeting Version 8.1.X – mainly because I already had it downloaded!

The first place I like to look when deciding a target version for an install or an upgrade is the Atlassian JIRA release notes. This will give you all the details about what versions are released, and what new features and bug fixes were included in that release. I love how Atlassian does their release notes. One of my biggest pet peeves is when a company has “Generic bug fixes” as their release notes – I mean my job is to manage a system to tell you exactly what you fixed!

Second thing I consider is also a bit of advice I’ll give to any admins out there. As a personal policy, I never install a version that ends in “0”. If it has two zeros, the policy stands doubly so. Nothing against Atlassian, they make comparatively good software.

However, I’ve worked with software companies, and remember the mad rush that follows any software release ending in zero. There are always bugs, and a dash to fix them. It’s also, incidentally, why I put a blackout window a month before through two weeks after any software release, where no changes to JIRA or Confluence are allowed.

Directory Layout

Now that you’ve have a target in mind, comes the next step. JIRA has two main directories on the file system it cares about, the Install Directory and the Home Directory. Took me forever to understand the difference, and what does what. However, it’s a good thing to learn now, as it will save you a good bit of time in hunting down problems.

Install Directory

The Install directory is where the files that make JIRA run live. With the exception of two or three configuration files, you should not modify anything in here. And if you do, be sure to follow Atlassian’s documentation.

You will typically find this directory at /opt/atlassian/jira, and this one I suggest keeping to the default location.

Home Directory

Unlike the Install Directory, JIRA’s home directory is where everything that makes your JIRA instance unique lives. Here you will find details about how JIRA connects to the Database, any saved attachments, log files, exports, and the Index.

That is why I like to keep it on a separate drive. Considering what all is stored in the JIRA Home Directory, it will grow considerably over your instances lifetime. It is also important that you have it on the fastest hardware you can afford. The speed at which JIRA can access it’s index will have a significant impact on both re-indexing times and JIRA’s performance overall.

By default, you can find the JIRA home at ‘/var/atlassian/application-data/jira’. However, if you have installed it in another location, you can use the command below to find where it is (assuming you know where the JIRA install directory is).

cat <jira_install_dir>/atlassian-jira/WEB-INF/classes/jira-application.properties | grep "jira.home = " | cut -d ' ' -f 3 | tr -d '\r'

Documents…always with the Documents

If you haven’t realized it yet, there is a lot of reading in being an admin. This week we are going to be looking at two documents: one new one and one returning one.

Our first document is Atlassian’s guide to installing JIRA on Linux. It includes a checklist of things to check before you install, and a rundown of how to do it. For this I will be using the Installer, and installing it as a service.

You can also do an install from a tar.gz archive, but this is a more intense process, and honestly one I find more ideal for automated installations and upgrades than something for a human to do. There are people who will swear by it, and in fact sometimes there’s some bug in Linux or the installer that forces you to consider an archive install, but whenever possible I’d rather not complicate things. “Keep it stupid simple” was something my engineering professors would chant whenever possible.

Our next document is the Atlassian MySQL document from last week. Eagle-eyed Admins would have noticed that I missed a step of that doc last week. This was intentional, as you cannot complete this step until you’ve installed JIRA, so we’ll be revisiting that when the time comes.

Getting the Installer

To get the installer, I’ll go to the JIRA Download Page, the click All Versions. From there, I’ll navigate to my target version, then select “Linux 64 bit”. At this time I have the same options of downloading it to my local computer and transfering it, or using wget to download it directly to the server. Be sure if you do the latter, that the URL you copy ends with the file extension “.bin”. The link to the download should be the “Get started” button that lights up after you agree to the Atlassian Software License Agreement, if you wish to copy/past it to the server.

Once you download or transfer it to your server, you will need to modify the permission so you can run the file. You can do so with the following command:

chmod 755 ./atlassian-jira-software-*.bin

This command uses a wildcard, so it should work for all future versions of JIRA – or at least until Atlassian changes the naming convention on their releases.

Running the Installer

Now comes the bit we’ve all been waiting for. Running the installer. I’m not going to lie to you, this part is going to be dry. There’s no help for it.

Anyways, to start it, just type in the file into your command line. If you wish to run JIRA as a service, you will need to either run this as root or run this in sudo.

./atlassian-jira-software-8.1.3-x64.bin

The first question it will ask is if you really want to install JIRA here. Take a moment to confirm the JIRA package and version are correct. I intended to install JIRA Software 8.1.3, so this all checks out. Hit enter or “o” followed by enter to proceed. Press “c” then enter to cancel the installer.

The next options it gives us to to select the operation we want to proceed with. Our options are:

  1. Express Install – uses all the default options. As we want to change our home directory, this isn’t the option we want.
  2. Custom Install – This is for more advanced users, it allows you to customize where JIRA is installed, where it’s home directory is, what port it uses, etc. This will be our option today.
  3. Upgrade an Existing JIRA install – not applicable here, but this is the option people are probably more familiar with.

So, considering how we want to set this up, we’ll hit “2”, then enter

The next question is in regards to our Install Directory. We want to keep this default, so we can just hit “Enter”

After this the installer will ask for the location of our home directory. As we want to change this, I entered the location for it. The installer will create any parent folders it needs to make the final structure you put here, so I find it helps to match their default structure as much as possible. Enter your chosen directory, double check it (no really….it’s going to be a headache otherwise), and hit Enter. If you wish to keep the default, just hit “Enter” without entering anything.

Next JIRA will ask you which ports you want it to use. As JIRA does not run as root, it cannot run natively on ports 80 and 443. Unless you plan on running more than one JIRA instance on the same server, you can keep to the defaults here.

The next question will ask if you want to run JIRA as a service. What this means is two fold: first JIRA will start up as part of the server boot sequence, and second you can start/stop/restart JIRA using systemctl. So…the answer to this is “Of course!”, so you can hit enter.

The last question here before JIRA is installed is asking you to confirm all the settings you entered. Just run over them to make sure they are correct, then hit enter.

It is at this point the installer will extract the files, install everything where it is needed, and setup the service. All told this process takes about a minute. After it is done, you should see this.

We actually do not want to start JIRA yet, as we have one more step to complete before we can give it a first boot. So type “n” then enter. This will complete the installer run.

After the Installer

As stated, we need to install the MySQL JDBC connector into JIRA. Due to licensing constraints, Atlassian is not allowed to distribute this as part of their software, so this is a step we must take.

To download this, we’ll need to look at the MySQL page for this. As always, choose your download method for this and move it onto the server. I will download the compressed TAR Archive as the tools for that are pretty common on linux servers.

 wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.48.tar.gz

I will then need to decompress the file with the following command:

tar -xzvf ./mysql-connector-java-5.1.48.tar.gz

After I’ve extracted the files I’ll need to copy the given file into the JIRA Install Directory.

cp ./mysql-connector-java-5.1.48/mysql-connector-java-5.1.48-bin.jar <jira_install_dir>/lib/

After this, we can finally start JIRA on the server, and start working in a web browser to finish it’s setup.

systemctl daemon-reload
systemctl start jira
systemctl status jira

The first command tells the system controller to reload what services are available – that is to say tell it to find JIRA’s service description. The second command starts JIRA, and the last is to make sure the service is up and running correctly. I recommend we take a few seconds between 2 and 3 to let things start up.

If “systemctl status jira” comes up clean, you can run the next command to make sure it comes up on boot.

systemctl enable jira

You can also use the following command to follow JIRA as it boots up. In fact, I will usually do this anytime I need to restart the JIRA service.

tail -f <jira_home_dir>/log/atlassian-jira.log

Firewalls

If you immediately try to jump onto the server on JIRA’s port, you may notice it fails to connect. CentOS and RHEL both use firewalld. This is a fact I’ll often forget, and sit there scratching my head for five minutes before feeling like an idiot. This is an easy fix, we just need to add an exception in there to allow traffic to flow on port 8080. This will only be temporary, because as soon as we get JIRA configured at set up, we’ll work on a proxy to handle traffic so it runs on port 80/443.

To add the exception, use the following command as sudo or root:

firewall-cmd --zone=public --add-port=8080/tcp

As the Database is located on the same server as the JIRA server, we do not need to open up port 3306 for MySQL traffic. However, if you’ve set up MySQL to run on a seperate box from JIRA, you will need to open up this port. However, we should be good to go for this build.

Post Install Setup

Assuming the system has come up cleanly and you’ve configured our firewall, you should see the above screen when you use your web browser to navigate to 

http://<ip_addr_of_server>:8080

You will want to select the second option marked “I’ll set it up myself”, then “next”, which will take you to the appropriate wizard.

The next screen will ask you for some details on the database. Here you will select “My Own Database”, then enter the details. The database, username, and password will have been set during the “JIRA’fying this DB” section from last week’s post.

Click “Test Connection” to be sure everything is correct before hitting “Next” to proceed. This will setup the Database. Depending on your system speeds, this will take a few minutes, so you might want to go get some water.

The next step will be some customization for your instance. You will need to put a base URL that works for now. Once we have the proxy setup, we’ll need to change it, but it will work until that time. As for the Application title, I always prefer to go with something like “Company – JIRA” or “Company – Division – JIRA” as appropriate. Just something to uniquely identify this JIRA instance within your organization.

If this is a test instance, I also like to add the word “DEV” or “TEST” at the beginning, to further emphasize this is not Production.

The next screen will ask for your license key from Atlassian. Copy/Paste it from https://my.atlassian.com and click next to proceed.

The next step will ask you for a admin username and password. A word of note here: Even if you intend to use Active Directory or Crowd to manage users, you will still need at least one local admin account. The reason why is you cannot edit a directory from an account associated with that directory, even if that account has the correct Admin privileges This means you will need a local account to modify that directory.

After you setup your account, it will ask you whether you want to set up a mail server. You will want to set this up before you move to production, as this is the main way JIRA communicates issue updates. That being said, you do not have to do it now, so we’ll take a pass on it today.

Click finish, and JIRA will take a second while it finalizes some system settings. However, we are not done yet!

That’s right, we need to finalize our account setup. Here you will choose the language appropriate for you, then click “Continue”

Next it will ask you for an avatar. This really isn’t required, and you can click next right away, but why would you say no to this friendly little ghost?

And finally JIRA will ask you if you want to setup a first project. Your options are:

  • See a project in action – This will create a JIRA project with some issues, so you can see how one looks when it is actively being used.
  • Create a new project – This will create a blank JIRA project from one of the default templates.
  • Import from another tool – This will allow you to import from another tool, like Bugzilla, Github, etc

As you can see, you have a good few options here. At the very least, you should be able to get something into a CSV format. I’ve done plenty of these, and can say that it will work fairly well so long as you have good clean data.

And that’s it for now!

We’ve gone from a blank server to a functioning JIRA instance! While this is definitely a milestone to celebrate, we are not quite done yet. Next week we’ll see what settings we’ll need to modify to get web traffic going through SSL.

This will require us to modify system files, setup Apache, and change some settings with JIRA itself. That being said, if you’ve made it this far you shouldn’t have any problems moving forward.

One of the things I intend to release once I finish this series is a checklist you can use as you proceed to make sure you aren’t missing any steps. When I’m going an upgrade or other maintenance, I’ll always take time beforehand to prepare one, and print it out so I can check things off as I’m doing it. I’ll even include the pre-flight checklist to make sure we are good to go.

Next week we are taking a break from this project. The last two week’s post has inspired me, and I will take a look next week to see if you can run JIRA smoothly on one of the new Raspberry Pi 4’s. Just experimenting for experiment’s sake. After that we’ll return to this project with setting up the Proxy through Apache. So until then, I’m Rodney, asking “Have you updated your JIRA issues today?”

Installing JIRA from scratch, Part 1, Database Install and Setup

Rodney N.5 Comments

When I was leaving VMware, I discovered something interesting. My junior admin – who, in two weeks, would be the senior admin – had never done an install of JIRA from scratch.

Obviously, this would not do. Over the next two weeks, we did several intensive training sessions, where I’d do an install why he took notes. Eventually we tested the new docs by having him do the install solo while I watched. He passed -with some tweaks to the documents.

Looking online, there just isn’t to many details on how to do an install from front to back. Yeah, all the pieces are there, but it’s up to a new admin to put them together – which if you’ve never done it before, can be daunting. But I intend to fix that, starting here.

My original idea for this article was to do it from the ground up. That is, setting up the VM, installing the OS, installing the prereqs, configuring the DB, installing JIRA and then the proxy. Upon reflection, I think it’s safe to assume a few things. First, you know how to setup a VM and install an OS…or at least, you know who to ask. Second, you don’t need 100% hand holding. So I will be scaling this article back some. If you are interested in how I go about setting up my VM’s and linux installs, let me know in the comments and I’ll write up that article.

OS Selection

Though I’m assuming you know how to install an OS, there is a myth that I see often: that you must run JIRA on Linux. Let me squash this right now.

IT DOESN’T MATTER WHAT OS YOU SELECT!

My advice is to use what you know. You don’t know linux but you can run a windows server? Use that! You want to run it on a Mac in the corner? Well…that might actually be an issue. It will run, but your support from Atlassian may not be there.

I’m not saying you can ignore the supported platforms document. You should double check with your choices here. But I am saying when your choices are as varied as they are in the doc, you have options.

To me, it is much more important you be familiar and comfortable with the operating system than following the crowd. Java, by it’s design, will take care of making sure it can run on your system. I started out as a Linux Admin, and can configure and tune Linux in my sleep. So I use what I know. But you don’t have to.

That being said, with only a few considerations, you can migrate an instance from one OS to another with minimal headaches. It’s not the easiest operation ever, but can be done well enough in a change window. Just something to keep in mind.

System Setup

Okay, now that we have that out of the way, I’m going to put down a few details about how the system is setup and what assumptions I am making.

JIRA System Specs

  • OS: CentOS 7 Linux
  • Database: Latest Supported MySQL (5.7 as of the writing of this article)
  • RAM: 4 GB
  • Processors: 2
  • Root Hard drive of 30 GB
  • Extra Hard drive of 50 GB to house the database and JIRA’s home directory

So a few things I want to note here. First off, I chose four gigabytes of ram for a reason. The way I like to break down a new system is like this: 2 GB (ish) for JIRA’s Heap memory, 1 GB of Ram for MySQL, and 1 for system resources. This usually gives you enough room to grow for a bit before it gets to be a problem.

Second, I like to keep a 2:1 ratio between GB of Ram and Processors. This isn’t listed anywhere as a best practice for VM’s, just something I’ve always done, and I’ve never had a cause for change the practice. When it does cause something to happen, I may re-evaluate the practice then.

Lastly, I like to keep the JIRA Home and Database on a separate drive from the system resources. This makes monitoring easier, as these will be the two directories that will grow the most as your JIRA instance matures. It’s strictly speaking not required, but I like to think it helps in the long run, even if it is more work up-front. For my system, I’ll have it mounted to /archive.

MySQL Database Setup

JIRA, at it’s heart, is a web based front end for the Database. This makes your database choice pretty important. As of this writing, JIRA supports the following database systems:

  • PostgreSQL 9.4, 9.5, and 9.6
  • MySQL 5.6 and 5.7
  • Oracle 12cR1
  • Microsoft SQL Server 2012, 2014, and 2016
  • Azure SQL

As with the OS Selection, it’s important to use what you know. I’ve used PostgreSQL and Microsoft SQL Server in systems before, but I’ve developed queries and databases for – as well as ran and tuned as a system – MySQL. Therefore, outside of concerns for performance, I’ll use what I know best.

Slow down there!

Take a moment and think not only about the system as you currently envision it, but as you will use it years down the road. Unlike OS Selection, JIRA is very much a pain to migrate from one Database to another, requiring you to do an XML export and import on an entirely new instance on the new DB system. It may get to the point before you realize it that it becomes wholly impractical to do so.

As such, consider where you see this instance in 3 years. If you plan for this to be used by a lot of people simultaneously, PostgreSQL might be the way to go. There are use cases for Oracle, MS SQL Server, and Azure as well, all depending on how you want to use it. Now is the time to just take a second and think about it. Your future self will thank you.

Documents

While we are slowing down, there are a few documents we will need to check out.

The first document we’ll need is from MySQL. CentOS ships with MariaDB in it’s repos. Normally that’d be fine, and in some cases even preferred, but JIRA expressly doesn’t support MariaDB. As such, we’ll need to add the Yum repos for MySQL to our soon-to-be JIRA server.

Next, there’s Atlassian’s guide to setting up a MySQL database for use with JIRA. This will tell us what accounts to create, settings to set, and things to do to get MySQL running smoothly with JIRA.

These are mostly compatible with the settings you need for Confluence, Bitbucket, and Bamboo, so you can centralize the database if you so wish. However, that also introduces a huge single point of failure for your whole system, so I don’t really recommend it.

Yeah, but your scientists were so preoccupied with whether or not they could, they didn’t stop to think if they should

Dr. Ian Malcolm, Jurassic Park, 1993

Getting Started

To start with, we’ll need a way to manage yum repositories, so the first step should be to install yum-utils. This will come in handy in a few later steps, so might as well take care of it now. I will be running this as root via the esxi interface (I disable root from remote connections), but use sudo where you need to. 

yum install yum-utils -y

As stated in the MySQL Document, we need to do is get the RPM for the MySQL Yum Repo onto the server. You can do this by downloading it to your local system and transferring it via scp/sftp/ftp/séance with the dead. But why. I prefer to get the link to the actual RPM (may take a couple of menus!), then use a utility called wget to retrieve it directly to the machine.

wget https://repo.mysql.com//mysql80-community-release-el7-3.noarch.rpm

You are free to use the above command, but I can’t guarantee it will work forever, so I always suggest you double check that link against MySQL’s website and renew it as needed.

Now that we have it downloaded, we do a yum localinstall against it. This will setup Yum to have the MySQL repo.

yum localinstall ./mysql80-community-release-el7-3.noarch.rpm

So now we have the Yum Repository for MySQL installed, but we still have a problem. MySQL will default to the latest available (MySQL 8) unless we tell it otherwise. As such, we first need to see what is enabled and not. To do so, we’ll use the following command:

yum repolist all | grep mysql

We should see the following output:

Here we see three repos enabled.

  • mysql-connectors-community/x86_64
  • mysql-tools-community/x86_64
  • mysql80-community/x86/64

To disable this and enable mysql57-community, we run the following two commands, in order

yum-config-manager --disable mysql80-community
yum-config-manager --enable mysql57-community

After you run these, re-run the yum repolist command to confirm the correct repo is enabled:

Main Event Time

Install the mysql-community-server and all it’s prerequisites with the following command:

yum install mysql-community-server -y

I know this seems anticlimactic considering how much you have to do to just get here. But that is one of the beautiful things here – it’s mostly automated. The install will take a second, but not terribly long in the grand scheme of things. Once done, start the MySQL server with the command below.

systemctl start mysqld

Wait a few seconds, then query the log for the temporary password. You’ll need it here in just a second, so might as well grab it.

grep 'temporary password' /var/log/mysqld.log

Take this password and run the command:

mysql_secure_installation

This is a script that will guide you through locking down your MySQL instance. You will need the temporary password we just got, but you will get a chance to setup a proper root password. Please be sure to set your MySQL root password to something unique and complex. You don’t want to be the one to be hacked because your DB root password was the same as your user password.

For the rest of the questions on this script, select Yes. This is just a fast way to do all the good hygiene stuff you should do when you make a new MySQL DB.

From here, we can run one more command to make sure MySQL comes up as part of the system’s boot sequence, and we are ready to start configuring the Database for JIRA.

systemctl enable mysqld.

JIRA’fying this DB

Here’s where we’ll start following the Atlassian Document I referenced earlier. Log into the machine, either via a console or ssh, and connect to MySQL using the following:

mysql -u root -p

Here you will use the Database Root password you just set within the Secure Install script. Here we’ll run a number of queries to setup the database and user JIRA will connect through.

CREATE DATABASE jiradb CHARACTER SET utf8mb4 COLLATE utf8mb4_bin;

Yes, you do need to set it up as utf8mb4 with a collation of utf8mb4_bin. Not using these will cause JIRA to throw errors, and in some cases fail altogether. This will setup the Database with MySQL. Note that there is no requirement to call the DB “jiradb”. You are free to use any name here, though I’d recommend you use one that is readily identifiable.

Next, we’ll setup the user using this next two queries.

GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,DROP,REFERENCES,ALTER,INDEX on <JIRADB>.* TO '<USERNAME>'@'<JIRA_SERVER_HOSTNAME>' IDENTIFIED BY '<PASSWORD>';
flush privileges;

In the GRANT statement above, we’ll replace the following as such:

  • <JIRADB> is whatever you named the database above
  • <USERNAME> is the username JIRA will use to connect to the DB
  • <JIRA_SERVER_HOSTNAME> to the name of the JIRA server (localhost in our case)
  • <PASSWORD> to the password JIRA will connect with

It’s important to security that the JIRA password be different from the root password. Don’t be that guy. Please do not be that guy.

After this, we are safe to exit MySQL (run the query ‘exit;’), and make some changes to the MySQL configuration file. This one…is very poorly named, considering history, but you can find it at /etc/my.cnf.

Yes, poorly named….

Anyways, it will ask you to make a number of changes. For the sake of brevity, I will include a copy of the config file that shall not be said aloud:

[mysqld]
datadir=/archive/mysql
socket=/var/lib/mysql/mysql.sock

symbolic-links=0

log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

#<---- Start JIRA Specific Settings ---->
default-storage-engine=INNODB
character_set_server=utf8mb4
innodb_default_row_format=DYNAMIC
innodb_large_prefix=ON
innodb_file_format=Barracuda
innodb_log_file_size=2G
#<---- End JIRA Specific Settings ---->

Note that I’ve changed the datadir to the /archive mount I mentioned before. Not necessary, but I still feel it will be beneficial long-term. Just a note, if you also intend to move the datadir in your MySQL install, and you are running on CentOS or REHL, you will need to adjust a few Selinux policies. Just run the following commands:

yum install policycoreutils-python -y
semanage fcontext -a -t mysqld_db_t "<new_datadir>(/.*)?"
restorecon -R <new_datadir>

Edit: Future Rodney stepping in here. It seems I missed recording a few steps, so I wanted to add them.

First, if you are moving the datadir in the mysql config file, you will also need to move the physical files as well. You can do so with the following command:

rsync -av /var/lib/mysql <new_datadir>

Second, after making any changes to the MySQL config file, be sure to restart MySQL with the following. Be sure you do the file move first, as well as the Selinux policy step if you are on RHEL/CentOS.

systemctl restart mysqld

Now returning you to your regularly scheduled blog post.

And there we are!

We’ve now setup a database and configured it for Linux. I think we are going to call it here (wow this one is long) and move on to installing JIRA next week. I also have a crazy plan to see if I can install this on a Raspberry Pi 4…while not supported, I do think it *might* be possible! Of course if I do achieve that, I’ll write up a how-to on that one as well! Until then, I’m Rodney, asking “Have you updated your JIRA issues today?”