Monday, May 9, 2016

Installing Solr

Blacklight is an interface for searching a Solr index. The quick install of Blacklight includes the installation of Solr, but we will be installing Solr manually instead. This gives us more control over which version of Solr to install.

If you are not familiar with Solr, imagine a book with an index at the back. If you are looking for a certain word or phrase in the book, you can spend lots of time scanning each page individually or you can flip to the index where the work has already been done for you. A Solr index is essentially the same thing as a book index.

The Solr index looks though all digital documents you tell it to, and indexes the words and phrases in the documents. When you want to search for a particular word, searching the Solr index is then much, much faster than a program that searches every document individually until it finds all instances of the word.

An additional benefit of Solr indexing is that the indexed words and phrases can be used as facets in faceted searching. The words and phrases are already grouped during the indexing process, so these groups are easily displayed and used to filter search queries.

If you have never installed or used Solr before, I recommend doing the Solr quickstart exercise to become familiar with some of the command line commands, what the interface looks like, and for a basic understanding of how the index works.

Installing Solr on the VPS

We will install Solr under the single user's directory, but we need to be logged in as the root user to do this.

1. Open a new Terminal window.

2. Connect to the VPS via ssh as the root user:

$ ssh root@example.com

3. Enter your root user's password when prompted for it and press Return.

4. Change directories to your single user's home directory:

# cd /home/username/

5. List the contents of this directory to check for an existing directory named "solr" with the following command:

# ls -a

6. If there is already a directory named "solr" then change directories into that folder:

# cd solr

If there is not a directory listed with the name "solr" then create the directory with the following command:

# mkdir solr

If you receive a permissions error saying you cannot make this directory, the easiest thing I found to do is to open a web browser, go to your cPanel on the Bluehost website, go to the file management section and click on the "File Manager" icon, navigate to the Home directory (/home/username/), and then create the folder using the File Manager interface.

7. In Terminal, once you have confirmed that the /home/username/solr directory exists, change directories into it:

# cd /home/username/solr

8. I used version 5.3.0 of Solr during my testing phases, so that is what I will use for the app. If you want to use a more recent version of Solr, check for the appropriate download link and substitute it in the command below.

Download the Solr tgz package to the /home/username/solr directory with the command:

# wget http://archive.apache.org/dist/lucene/solr/5.3.0/solr-5.3.0.tgz -O solr-5.3.0.tgz

If successful, you will see something similar to the following in the Terminal window, followed by a new command line prompt:

--2016-05-11 14:04:09--  http://archive.apache.org/dist/lucene/solr/5.3.0/solr-5.3.0.tgz
Resolving archive.apache.org... 163.172.17.199
Connecting to archive.apache.org|163.172.17.199|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 134914712 (129M) [application/x-gzip]
Saving to: “solr-5.3.0.tgz”

100%[===========================================================================================>] 134,914,712 18.3M/s   in 8.6s   

2016-05-11 14:04:20 (14.9 MB/s) - “solr-5.3.0.tgz” saved [134914712/134914712]


10. Unpack the tgz package with the following command:

# tar -xzvf ./solr-5.3.0.tgz

The Terminal window will print many lines of text showing the contents of the tgz package being unpacked into the solr directory.

11. Once the unpacking is finished and you see a new command line prompt, change directories into the solr subdirectory (in our case it is for version 5.3.0, which will be different for you if you installed a different version):

# cd solr-5.3.0

12. Start up your solr service with the following command:

# bin/solr start -noprompt

If successful, you will see:

Waiting up to 30 seconds to see Solr running on port 8983 [\] 
Started Solr server on port 8983 (pid=912). Happy searching!


13. Check if your solr service is running and accessible by opening a web browser and going to your URL followed by the 8983 port number:

http://example.com:8983

If everything is working properly, you should see the Solr admin interface.

14. If you want to free up space on the server, you can now delete the .tgz file you downloaded.


Previous post: Setting up the MySQL database

Next post: Installing Passenger

Saturday, May 7, 2016

Setting up the MySQL database

The Blacklight app uses a sqlite3 database by default, but the Blacklight project recommends using a MySQL or PostgreSQL database for a production deployment. The CentOS VPS server through Bluehost already has MySQL installed, so we will use that for our production database.

Side note: I did try to install and use the PostgreSQL database, but ran into several problems along the way. This prompted me to go with MySQL instead since it was already installed and mostly configured.

Setting up the MySQL database for the app

1. Go to your Bluehost cPanel and find the "Database tools" section. In this section, click on the "MySQL Databases" icon to create the database and database user.

 

2. On the next screen, find the "Create a New Database" form and type in the name of your database. You can name it whatever you like - I'll call it "test-app_production" in this example. Then click the Create Database button. You'll most likely be taken to a confirmation screen. From there, go back to the MySQL Databases page using the link provided.


3. Next, go to the "MySQL Users" section to create the database user. Fill in the form with a username and password. Then click the "Create a User" button. You'll most likely be taken to a confirmation screen again. From there, go back to the MySQL Databases page using the link provided.


4. Go to the "Add a User to a Database" section and select the correct username and database from the dropdown menu. Then click the "Add" button.

 

5. You will be taken to a page where you can select the privileges for this user. Select "All Privileges" and then click the "Make Changes" button. Once you see the confirmation that the changes were saved, go back to the MySQL Databases page with the link provided.



6. Head back to your cPanel next. We need to make a change to the collation of the database before we're done with MySQL setup. Find the database tools section again and click on the phpMyAdmin icon. This will take us to the phpMyAdmin interface, where we can make changes to the structure and contents of the database.


7. In the phpMyAdmin interface, click on the Databases tab to see a list of your databases.


8. Click on the database you just created (in our case, it is "test-app_production").


9. Click on the "Operations" tab.


10. Select "utf8_bin" in the Collation select menu and click save. The default "latin1_swedish_ci" won't work with the Blacklight installation. The "latin1_swedish_ci" collation does not support all of the characters Blakclight will be inserting into the database tables, so if we don't change this we will get an error during the database migration step later on.


11. After the new collation is saved, close out of the phpMyAdmin interface. Your MySQL database should now be configured for use with the Blacklight app.


Previous post: Installing Rails

Next post: Installing Solr

Thursday, May 5, 2016

Installing Rails

1. Before we move forward with installing Rails, take a moment to check for updates to the Ruby gems with the following command:

# gem update

2. And then check for updates to the RubyGems software itself with the following command:

# gem update --system

3. Installing Rails is as easy as typing the following into the system console and pressing Return:

# gem install rails -v 4.2.6

The -v part of the command allows you to choose which version of Rails you would like to install (we installed 4.2.6 here). The system will install the Rails gems, which will take a few minutes or more.

4. Once Rails is successfully installed, use the following command to check the Rails version:

# rails -v

The system console should respond:

Rails 4.2.6

If it does, you have successfully installed Rails!


Previous post: Installing Java Development Kit (OpenJDK)

Next post: Setting up the MySQL database

Installing Java Development Kit (OpenJDK)


Blacklight requires a Java Runtime Environment of 1.7 or higher. We will install the OpenJDK developer package on the server. If you would like to install the Oracle JDK, Oracle has instructions published on its JDK installation page. I found the OpenJDK installation instructions for Red Hat Linux on the OpenJDK site here.

1. In the system console, type the following and press Return:

# yum install java-1.8.0-openjdk-devel

The system console will ask you to confirm the installation during the process.

2. Once OpenJDK successfully installs, you can check the java version with the following command:

# java -version

The system console should respond with

openjdk version "1.8.0_91" OpenJDK Runtime Environment (build 1.8.0_91-b14) OpenJDK 64-Bit Server VM (build 25.91-b14, mixed mode)

If you see, this - congrats! OpenJDK is installed.


Previous post: Installing Node.js and EPEL

Next post: Installing Rails

Wednesday, May 4, 2016

Installing Node.js and EPEL

To install Node.js on our CentOS server, first run the following command (I adapted these instructions from the Phusion Passenger installation tutorial, which we continue with later):

sudo yum install -y epel-release yum-utils

Next, run the following command to install Node.js

sudo yum install -y --enablerepo=epel nodejs npm



Previous post: Installing Bundler

Next post: Installing Java Development Kit (OpenJDK)

Installing Bundler

The next step is to install Bundler to manage the Ruby gems used in Ruby on Rails apps. It can be installed in a single step at the command line:

# gem install bundler

Previous post: Installing Ruby

Next post: Installing Node.js and EPEL

Installing Ruby

Now that RVM is installed and working at the command line, you can move forward with installing Ruby on the server.

1. Check to see which, if any, version of Ruby are installed by typing the following and pressing Return:

# rvm list known

Most likely, no Ruby installations will show up yet.

2. If you do see some Ruby installation(s), you can check the default version currently in use by typing the following and pressing Return:

# ruby -v

3. To install a specified version of Ruby, type the following command and press Return (change the version number if you would like something other than Ruby 2.2.4):

# rvm install 2.2.4

The installation will begin. It may take a while (mine took between 5-10 minutes).

4. Once RVM installs the version of Ruby you specified, check the default version of Ruby again by entering:

# ruby -v

The console should respond with:

ruby 2.2.4p230 (2015-12-16 revision 53155) [x86_64-linux]

or something similar depending on which version you installed.

5. If the default version is not the version you just installed, and you would like to use the newly installed version, enter:

# rvm use 2.2.4

6. If the new version is not the default and you would like to make it the default, enter:

# rvm use 2.2.4 --default

7. Update your Ruby gems

# gem update

Congrats! You successfully installed Ruby and updated the Ruby gems. You can use the commands above to install new versions in the future, switch between versions, and change the default version of Ruby.


Previous post: Installing RVM to manage and install Ruby

Next post: Installing Bundler


Installing RVM to manage and install Ruby

The most appropriate installation method depends on your operating system and whether you will install more than one version of Ruby. Ruby installation can be done with package management systems, installers, managers, or building from source code. A list of available installation tools/methods can be found on the Ruby website.

The Linux Red Hat CentOs is the operating system on the Bluehost VPS. I decided to use Ruby Version Manager (RVM) to install and manage Ruby. It is compatible with Linux, assists with easy installation, and provides easy management of upgrades to newer version of Ruby in the future.

Install RVM

1. First go to the Terminal window that is connected via ssh to the VPS (see previous post for help with this). Once you are connected, make sure you are in your user directory by running the following command (Don't type the # symbol - that just represents the end of the command line prompt):

# pwd

Hopefully you will see the following response (where "username" is you actual username):

/home/username

If you do, you are in your user's home directory. If you don't, you can change to that directory using the cd command:

# cd /home/username

2. Once you confirm you are in your user's home directory, type the following and hit Return:

# gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3

This will retrieve the public key for the install. You will see something similar to the following as the key is retrieved:

gpg: requesting key D39DC0E3 from hkp server keys.gnupg.net
gpg: key D39DC0E3: "Michal Papis (RVM signing) <mpapis@gmail.com>" not changed
gpg: Total number processed: 1
gpg:              unchanged: 1


3. Once the public key is retrieved, you will see the command line prompt again. Now type the following and hit Return to install the latest stable version of RVM:

# \curl -sSL https://get.rvm.io | bash -s stable

Installation status messages similar to the following will appear in the Terminal window as RVM is installed:

Downloading https://github.com/rvm/rvm/archive/1.27.0.tar.gz
Downloading https://github.com/rvm/rvm/releases/download/1.27.0/1.27.0.tar.gz.asc
gpg: Signature made Tue 29 Mar 2016 01:49:47 PM UTC using RSA key ID BF04FF17
gpg: Good signature from "Michal Papis (RVM signing) <mpapis@gmail.com>"
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 409B 6B17 96C2 7546 2A17  0311 3804 BB82 D39D C0E3
     Subkey fingerprint: 62C9 E5F4 DA30 0D94 AC36  166B E206 C29F BF04 FF17
GPG verified '/home/username/.rvm/archives/rvm-1.27.0.tgz'

Installing RVM to /home/
username/.rvm/
    Adding rvm PATH line to /home/
username/.profile /home/username/.mkshrc /home/username/.bashrc /home/username/.zshrc.
    Adding rvm loading line to /home/
username/.profile /home/username/.bash_profile /home/username/.zlogin.
Installation of RVM in /home/
username/.rvm/ is almost complete:

  * To start using RVM you need to run `source /home/
username/.rvm/scripts/rvm`
    in all your open shell windows, in rare cases you need to reopen all shell windows.

#
username,
#
#   Thank you for using RVM!
#   We sincerely hope that RVM helps to make your life easier and more enjoyable!!!
#
# ~Wayne, Michal & team.

In case of problems: https://rvm.io/help and https://twitter.com/rvm_io

  * WARNING: You have '~/.profile' file, you might want to load it,
    to do that add the following line to '/home/
username/.bash_profile':

      source ~/.profile


  * WARNING: you have GEM_HOME="/usr/local/rvm/gems" this is conflicting with RVM, make sure to:

      unset GEM_HOME




Note: These instructions can be found on the RVM website home page. This is the simple installation. If you want to install using a more secure installation, you can find instructions on the RVM site here. There are also other installation options, such as installing Ruby and/or Rails with RVM, that are outlined on the RVM site here.


4. Once the installation successfully completes, you need to reboot the server before you can use RVM. There are two ways to reboot the server.

The first way is to press the "Send CtrlAltDel" button at the top right part of the system console window:


The second way is to go to the "server management" page in your Bluehost account interface and click the "reboot" button:



5. After the server shuts down and restarts (this will probably take a couple of minutes), you will need to log in again. Close your Terminal window and open a new one. Connect to the VPS via ssh again, and then run the following command to see if RVM is now available to use at the command line:

# type rvm | head -n 1

The system console response should be

rvm is a function

If it is, then you successfully installed RVM and made it available to use as a command line function!

Troubleshooting


When I installed RVM the first time, I installed it as the root user. That is not preferred, as written on the RVM site. It is better to install it as a single user in a user account's home directory as the steps above describe.

Because I initially installed it for multiple users as root, I ran into some problems and needed to uninstall and reinstall from scratch. To do this, I found the best method requires several steps (I found this explanation on this stackoverflow thread):

1. First run the following command (you might have to do this as the root user if you installed it as the root user):

gem uninstall rvm

2. After the uninstaller finishes running, there may be a couple of other files to find and delete manually. Check for mentions of rvm in the PATH variable and remove them if they are there. Also, look for rvm in the .bashrc, .profile, and .bash_profile files. Remove any mentions you find in those files. Lastly, look for files in the following locations and delete them if they exist:

/etc/rvmrc
~/.rvmrc

If I remember correctly, after going through these steps I was able to install RVM from scratch in the single user's home directory without any errors.



Previous post: Connecting to the VPS through Terminal and why I prefer this over the System Console

Next post: Installing Ruby



Tuesday, May 3, 2016

Connecting to the VPS through Terminal and why I prefer this over the System Console

Now that you successfully connected to the VPS through the System Console on the Bluehost web admin interface, we will connect through the Terminal program on a Mac.

There are probably many reasons Terminal would be preferred over System Console, but several things possible in Terminal, and not possible in System Console, include:

  • The ability to scroll up and down in the Terminal window to see previous actions no longer in view
  • The ability to copy and paste text from Terminal to save for later
  • The ability to have multiple Terminal windows open, all connected to the VPS (this is helpful for running actions as different users and for troubleshooting)

Connect to the VPS using Terminal


1. Open a Terminal window. On a Mac the program is in Applications>Utilities>Terminal and has the icon:


A Terminal window similar to the following will open:


2. ssh into the VPS account on the server with the following command:

$ ssh username@example.com

3. Enter the password when prompted. (You will not be able to see the characters as you type)

4. If successful, you will probably receive a "Last login" message and your new command line prompt in Terminal will become similar to

username@server.example.com [~]#

If this is what you see, you have successfully connected to the VPS using the command line in Terminal.

From this point forward, I will be running the commands in Terminal instead of the System Console.


Previous post: Connecting to the VPS server via the Bluehost system console

Next post: Installing RVM to manage and install Ruby

Monday, May 2, 2016

Connecting to the VPS server via the Bluehost system console

Once the VPS hosting plan is activated with Bluehost, you will have access to cPanel (just like on a shared hosting plan) as well as root access to the files on the server (not available on a shared hosting plan). Most importantly, you will be able to access the virtual server via the command line through a system console.

Accessing the server through the system console will help by allowing us to install the necessary components for the app using the command line.

Access the system console


1. Log in to your Bluehost account and click on the "hosting" section of your account page.


2. Click on the "performance" section link in the sub-menu, just to the right of the cpanel page link.


3. Before you can use the system console, you need to set a root password. To set the root password (or update it if you want to change it), go to the "access management" section by clicking on the section link in the left sidebar menu. Next, locate the password section on the Root tab of the access management section. Enter a password or use the password generator to create a root password. Memorize this password or write it down because you will need it to log into the system console.


4. After the root password is set, click on the "system console" section link in the left sidebar.


5. When you click on the "system console" section link, the system console will automatically open in a new browser tab or window. It will look something like this:


6. Log in to the system console by first typing 'root' for the login username and pressing Return.

7. When prompted for the password, type in the root password you created and then press Return. (You won't see your password characters as you type them)

8. That's it! If you were successful, you will see a command line style prompt similar to the following (with "[your-domain-name]" replaced by your actual domain):

;root@server.[your-domain-name]:root@server.[your-domain-name] [~]# _

If this is not your first time logging in, you will also probably see a "Last login" message between the password line and the command line prompt.

Troubleshooting

If you run into problems with setting your root password, you may need to contact Bluehost to make sure you have the administrative privileges to do so. If you are the account holder, they should be in place by default, but if someone added you as an additional user you may need to update some privileges.

I didn't run into any issues with the system console window automatically opening, but I imagine there may be an issue with pop-up blockers if you have strict pop-up settings activated in your browser.

Remember to use 'root' (without the quotation marks) to log into the console - not your Bluehost account login name.


Previous post: Introduction

Next post: Connecting to the VPS through Terminal and why I prefer this over the System Console



Introduction

In this series of blog posts, I will write about my experience installing the Blacklight search app on a virtual private server (VPS) hosted by Bluehost.

Blacklight is built on the Ruby on Rails framework. While installing Blacklight on a local computer and running it on the local computer's server is relatively simple, I have not found a detailed tutorial describing the start to finish process for installing it on a commercial hosting service server. Bluehost does not offer technical support for Ruby on Rails, but they do have a few pages on their website that offer piecemeal information that comes in helpful when combined with other resources on the Web.

These blog posts will describe in as much detail as possible the steps to successfully install not only a Ruby on Rails app, but specifically a Blacklight search app, on a VPS hosted by Bluehost.

Pre-installation

Before getting started with the install, you will need to set up a VPS hosting service with Bluehost. You can view the pricing plans and server details on this page. This installation is happening on the 12-month Standard VPS hosting plan. I imagine following this tutorial with a VPS from a different hosting service would work, but I can't promise it will be exactly the same.

If you haven't worked with Ruby on Rails before, I recommend working through one or more Ruby on Rails tutorials. Codecademy offers a free and fun introduction/tutorial for setting up a Ruby on Rails app.

I also recommend heading over to the Blacklight project's website and at the very least reading through their quickstart guide.

I am no expert at Ruby on Rails, but will hopefully pull this off using resources like these (along with some trial and error, of course).

Why VPS?

Great question. The initial attempt to create this Blacklight app was done on a shared hosting service plan with Bluehost. This did not succeed because Blacklight has minimum version requirements for Ruby, Rails, and Java.

The shared hosting plan does offer Ruby on Rails app creation through cPanel, but the version of Java installed on the shared servers was not high enough to run the Blacklight app. If you are just looking to create a simple Ruby on Rails app, you can do so on the shared hosting plan. However, if you are trying to create a specific app, such as Blacklight, that requires more up-to-date software, you will need to go with the VPS plan or a dedicated server plan. Bluehost assured me that we have enough control to install whatever versions of Ruby, Rails, and Java we need to run the app. Let's hope they're right!