14 min read

In this article by Jeroen van Baarsen, the author of the book GitLab Cookbook, we will see how we can manage GitLab. We look at procedures to update GitLab and also concentrate on troubleshooting problems faced.

  • Updating an Omnibus installation
  • Updating GitLab from a source installation
  • Troubleshooting your GitLab installation
  • Creating a backup
  • Restoring a backup
  • Importing Git repositories

(For more resources related to this topic, see here.)

Introduction

When running your GitLab installation, you need to update it every once in a while. GitLab is released every 22nd day of the month, so around the end of the month would be a nice time to update! The releases on the 22nd day are well tested, as gitlab.com is using the release candidates in production all the time. This way, you can be sure that the release meets the standards of the GitLab team!

In this chapter, we will take a look at how you can update your GitLab server and how you can create backups and restore them.

If you want to know what has changed in the new release, you can take a look at the change log provided in the repository for GitLab at https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG.

Updating an Omnibus installation

In this article, we will take a look at how you can update your GitLab installation when you install it via the Omnibus package. In this article, I’ll make the assumption that you’re using Ubuntu 12.04. You can use other Linux distributions; the steps for these can be found at about.gitlab.com.

How to do it…

Let’s update the Omnibus installation with the following steps:

  1. Log in to your server via SSH.
  2. Stop the unicorn server: 
    $ sudo gitlab-ctl stop unicorn
  3. Stop the background job server: 
    $ sudo gitlab-ctl stop sidekiq
  4. Create a backup in case the upgrade fails: 
    $ sudo gitlab-rake gitlab:backup:create
  5. Download the package from the GitLab website (https://about.gitlab.com/downloads/): 
    $ wget https://downloads-packages.s3.amazonaws.com/ubuntu-12.04/gitlab_7.1.1-omnibus.1-1_amd64.deb
  6. Install the new package (change the x.x.x part to the correct version number from your download): 
    sudo dpkg -i gitlab_x.x.x-omnibus.xxx.deb
  7. Reconfigure GitLab: 
    sudo gitlab-ctl reconfigure
  8. Restart all the services: 
    sudo gitlab-ctl restart

How it works…

On the 22nd day of every month, a new version of GitLab is released. This also includes the Omnibus package. As the installation of Omnibus based on GitLab does not take very long, the GitLab team has decided to install a new version of GitLab and preserve the old data of the old installation; this way, you don’t go over an update process, but you will be guided through the installation process as if you’re installing a new GitLab instance.

So, when you’re updating an Omnibus-based installation, you’re not really updating but rather installing a newer version and reconfiguring it to use the old data.

One thing you have to keep in mind is that making backups is very important. As any update can go wrong at some level, it’s a good feeling to know that when stuff goes wrong, you always have a backup that you can use to get back up and running as quickly as possible.

Updating GitLab from a source installation

Updating used to be a lot of work; you had to open the update document to find out that you need to perform about 15 steps to upgrade your GitLab installation.

To tackle this issue, the GitLab team has created a semiautomatic upgrader. When you run the upgrader, it will check whether there is a new minor version. If there is, it will start the upgrade process for you. It will perform database migrations and update config files for you.

How to do it…

Upgrade your source installation with the following steps:

  1. Log in to your server using SSH.
  2. We start with creating a backup just in case something goes wrong.
  3. Go to the folder of your GitLab instance: 
    $ cd /home/git/gitlab
  4. Create the backup; this might take a little while depending on the amount of repositories and the size of each individual repository: 
    $ sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production
  5. Stop the server: 
    $ sudo service gitlab stop
  6. Run the upgrader: 
    $ if [ -f bin/upgrade.rb ]; then sudo -u git -H
    ruby bin/upgrade.rb; else sudo -u git -H
    ruby script/upgrade.rb; fi
  7. Start the application: 
    $ sudo service gitlab start && sudo service nginx restart
  8. Check whether everything went fine. We can use the self-check GitLab ships with: 
    $ sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production
  9. If the GitLab check indicates that we need to upgrade the GitLab shell, we can do this by performing the following steps.
  10. Go to the shell directory: 
    $ cd /home/git/gitlab-shell
  11. Fetch the latest code: 
    $ sudo -u git -H git fetch
  12. Run the following command to set the pointer to the latest shell release. Change 1.9.4 to the current version number. You can find this number in the output of the check we did in step 9: 
    sudo -u git -H git checkout v1.9.4

How it works…

It is highly recommended that you create a backup before you run the upgrader as every update can break something in your installation. It’s a good feeling to know that when all goes wrong, you’re prepared and can roll back the upgrade using the backup.

Troubleshooting your GitLab installation

When your GitLab instance does not work the way you expect it to work, it’s nice to have a way to check what parts of your installation are not working properly. In this article, we will take a look at the self-diagnostic tools provided by GitLab.

How to do it…

Learn how to troubleshoot your GitLab server with the following steps:

  1. Log in to your server using SSH.

    The first case shows troubleshooting in the case of a GitLab source installation.

  2. Go to your gitlab folder: 
    $ cd /home/git/gitlab
  3. To autodiagnose your installation, run the following command: 
    $ sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production
  4. When there is a problem with your GitLab installation, it will be outputted in red text, as shown in the following screenshot:
  5. The solution for the problem is also given; just follow the explanation given by the problem you walk into. In this case, we have to run the following command: 
    $ sudo-u git -h bundle exec rake gitlab:satellites:create RAILS_ENV=production
  6. If everything is green, your installation is in great shape!

    The next few steps concentrate on troubleshooting in the case of the Gitlab Omnibus installation.

  7. Run the following command: 
    $ sudo gitlab-rake gitlab:check
  8. When you have any problem with your GitLab installation, this will be outputted; also, the possible solution will be shown.
  9. The solution that is given for this problem is the solution for the source installation. To fix this issue in the Omnibus installation, we need to alter the command a little. You have to replace the $ sudo-u git -h bundle exec rake part with $ sudo gitlab-rake. So, the command will look as follows: 
    $ sudo gitlab-rake gitlab:satellites:create RAILS_ENV=production
  10. If everything is green, your installation is in top shape!
  11. In case you need to view the logs, you can run the following command: 
    $ sudo gitlab-ctl tail
  12. To exit the log flow, use Ctrl + C.

How it works…

When you think your GitLab installation might not be in a good shape or you think you’ve found a bug, it’s always a good idea to run the self-diagnostics for GitLab. It will tell you whether you’ve configured GitLab correctly and whether everything is still up to date.

Here is a list of what will be checked:

  • Is your database config correct?
  • If your database is still running SQLite instead of PostgreSQL or MySQL, it will give you a warning.
  • Are all the user groups configured correctly?
  • Is the GitLab config present and up to date?
  • Are the logs writable?
  • Is the tmp directory writable?
  • Is the init script present and up to date?
  • Are all the projects namespaced? This is important if you’ve upgraded from an old version.
  • Are all the satellites present?
  • Is your Redis up to date?
  • Is the correct Ruby version being used?
  • Is the correct Git version being used?

This is the first place you should start when walking into problems. If this does not give you the correct answers, you can open an issue on the GitLab repository (https://gitlab.com/gitlab-org/gitlab-ce). Make sure you post the output of this check as well, as you will most likely be asked to post it anyway.

Creating a backup

It’s important that you have your source code secured so that when your laptop breaks down—or even worse, the office got destroyed—the most valuable part of your company (besides the employees, of course) is still intact. You’ve already taken an important step into the right direction; you set up a Git server so that your source code has multiple places to live. However, what if that server breaks down?

This is where backups come into play! GitLab Omnibus makes it really easy to create backups. With just a simple command, everything in your system gets backed up: the repositories as well as the databases. It’s all packed in a tar ball, so you can store it elsewhere, for example, Amazon S3 or just another server somewhere else.

In this article, we not only created a backup, but also created a schema to automatically back up the creation using crontabs. This way, you can rest assured that all of your code gets backed up every night.

How to do it…

In the following steps, we will set up the backups for GitLab:

  1. First, log in to your server using SSH.

    The first few steps concentrate on the GitLab source installation.

  2. Go to your gitlab folder: 
    cd /home/git/gitlab
  3. Run the backup command: 
    bundle exec rake gitlab:backup:create RAILS_ENV=production

    The backup will now run. This might take a while depending on the number of repositories and the size of each repository.

    The backups will be stored in the /home/git/gitlab/tmp/backups directory.

    Having to create a backup by hand everyday is no fun, so let’s automate the creation of backups using a cronjob file.

  4. Run the following command to open the cronjob file: 
    $ sudo -u git crontab -e
  5. Add the following content to the end of the file: 
    0 2 * * * cd /home/git/gitlab && PATH=/usr/local/bin:/usr/bin:/bin bundle exec rake
gitlab:backup:create RAILS_ENV=production
  6. Save the file, and the backups will be created everyday at 2 A.M.

    The next few steps talk about the GitLab Omnibus installation.

  7. To create the backup, run the following command: 
    $ sudo gitlab-rake gitlab:backup:create

    A backup is now created in the $ /var/opt/gitlab/backups directory.

  8. Let’s verify that our backup is actually there: 
    $ ls /var/opt/gitlab/backups/

    You should see at least one filename, such as 1404647816_gitlab_backup.tar. The number in the filename is a timestamp, so this might differ in your case.

    Now we know how to create the backup. Let’s automate this via a cronjob file.

  9. Run the following command as the root user: 
    $ crontab -e
  10. Add the following code to the end of the file to have the backup run every day at 2 A.M.: 
    0 2 * * * /opt/gitlab/bin/gitlab-rake gitlab:backup:create

    After you save the file, a backup will be created every day at 2 A.M. This is great, but there is a tiny catch; if you have the backups run for too long, it will take up all of your disk space. Let’s fix this!

  11. Open this file location: /etc/gitlab/gitlab.rb.
  12. Let’s have the backups only last for a week. After that, they will be destroyed. 7 days is 604,800 seconds. Add the following code to the bottom of the file: 
    gitlab_rails['backup_keep_time'] = 604800
  13. To have the changes take effect, we have to tell GitLab to reconfigure itself. Run the following command: 
    $ sudo gitlab-ctl reconfigure

Restoring a backup

If your server breaks down, it is nice to have a backup. However, it’s a pain when it’s a full day’s work to restore that backup; it’s just a waste of time.

Luckily, GitLab makes it super easy to restore the backup. Retrieve the backup from your external Amazon S3 storage or just your external hard drive, copy the file to your server, and run the backup restore command. It won’t get any easier!

Getting ready

Make sure you have a recent backup of your GitLab instance. After you restore the backup, all the data created between the backup creation and the restoration of your backup will be lost.

How to do it…

Let’s restore a backup using the following steps:

  1. Start with a login to your server using SSH.

    The next few steps concentrate on the GitLab source installation.

  2. Go to your gitlab folder: 
    $ cd /home/git/gitlab
  3. Go to the backup folder of GitLab: 
    $ cd /home/git/gitlab/tmp/backups
  4. Look at the filename of the most recent file and note the number that the filename starts with.
  5. Go back to your GitLab folder: 
    $ cd /home/git/gitlab
  6. Now, run the following command and replace the 1234567 part with the number you took from the latest backup filename: 
    $ bundle exec rake gitlab:backup:restore RAILS_ENV=production BACKUP=1234567

    The next few steps concentrate on the GitLab Omnibus installation.

  7. Make sure your backup file is located at /var/opt/gitlab/backups: 
    $ cp 1407564013_gitlab_backup.tar /var/opt/gitlab/backups/
  8. Before we can restore the backup, we need to stop our instance. First, stop GitLab itself: 
    $ sudo gitlab-ctl stop unicorn
  9. Next, stop the background worker: 
    $ sudo gitlab-ctl stop sidekiq
  10. Now, we will restore our backup. You need to provide the timestamp of the backup you want to restore. The timestamp is the long number before the filename. Warning: this will overwrite all the data in your database! The following command depicts this: 
    $ sudo gitlab-rake gitlab:backup:restore BACKUP=TIMESTAMP_OF_BACKUP
  11. Restoring the actual backup might take a little while depending on the size of your database.
  12. Restart your GitLab server again: 
    $ sudo gitlab-ctl start

Importing an existing repository

It’s quite simple to import your repositories from somewhere else. All you need to do is create a new project and select the repository to be imported. In this article, we will take a look at how this is done. For this article, we will import the repository hosted on GitHub at https://github.com/gitlabhq/gitlab-shell.

How to do it…

In the following steps, we will import a repository:

  1. Log in to your GitLab instance.
  2. Click on New project.
  3. Enter the project name as GitLab Shell.
  4. Click on Import existing repository?.
  5. Enter the URL for the repository we want to import: 
    https://github.com/gitlabhq/gitlab-shell
  6. Now, click on Create Project.
  7. Importing the existing repository might take a while depending on the size of the repository.
  8. After the importing is done, you will be redirected to the project page.
  9. Let’s check whether it’s actually an imported repository. Click on the Network menu item. If everything is fine, you should see the graph in the following screenshot:

How it works…

There is really nothing magical about importing a repository. All GitLab does is clone the URL you give it to its own satellite. After this is done, the satellite will be linked to your project, and you’re done!

What if your repository is private and not publicly accessible? You can import it by adding the user credentials in the URL. Don’t worry; this information is not stored anywhere!

So, if we have the same repository as the one we used earlier but it has credentials, it will look like what is shown in https://username:[email protected]/gitlabhq/gitlab-shell.

Summary

In this article, we learned about the update processes for GitLab. We also got a gist of the ways that we can use to troubleshoot problems faced. The article explains how we can restore and back up an existing repository.

Resources for Article:

Further resources on this subject:

Packt

LEAVE A REPLY

Please enter your comment!
Please enter your name here