At 3scale we find Amazon to be a fantastic platform for running APIs due to the complete control you have on the application stack. However for people new to AWS, the learning curve is quite steep. So we put together our best practices into this short tutorial. Besides Amazon EC2, we’ll use the Ruby Grape gem to create the API interface and an Nginx proxy to handle access control. Best of all everything in this tutorial is completely free.
For the purpose of this tutorial you’ll need a running API based on Ruby and Thin server. If you don’t have one you can simply clone an example repo as described below (in the “Deploying the Application” section). If you’re interested in the background of this example, the Sentiment API, you can see a couple of previous guides which 3scale has published. Here we use version_1 of the API (“up and running in 10 minutes“) with some extra sentiment analysis functionality (this part is covered in the second part of the Sentiment API tutorial).
Now we’ll start the creation and configuration of the Amazon EC2 instance. If you already have an EC2 instance (micro or not), you can jump to the next step, Preparing Instance for Deployment.
Creating and configuring EC2 Instance
Let’s start by signing up for the Amazon Elastic Compute Cloud (Amazon EC2). The free tier is enough to cover all our basic needs. Once the account is created, go to the EC2 dashboard under your AWS Management Console and click on the Launch Instance button. That will transfer you to a pop-up window where you’ll continue the process:
- Choose the classic wizard
- Choose an AMI (Ubuntu Server 12.04.1 LTS 32bit, T1micro instance) leaving all the other settings for Instance Details as default
- Create a keypair and download it – this will be the key that you’ll use to make an ssh connection to the server, it’s VERY IMPORTANT!
- Add inbound rules for the firewall with source always 0.0.0.0/0 (HTTP, HTTPS, ALL ICMP, TCP port 3000 used by the Ruby Thin server)
Preparing Instance for Deployment
Now, as we have the instance created and running, we can connect there directly from our console (Windows users from PuTTY). Right click on your instance, connect and choose Connect with a standalone SSH Client.
Follow the steps and change the username to ubuntu (instead of root) in the given example.
After executing this step you are connected to your instance. We’ll have to install new packages. Some of them require root credentials, so you’ll have to set a new root password:
sudo passwd root. Then login as root:
Now with root credentials, execute:
sudo apt-get update
And switch back to your normal user with
exit command and install all the required packages.
- Install the libraries which will be required by rvm, Ruby and Git:
sudo apt-get install build-essential git zlib1g-dev libssl-dev libreadline-gplv2-dev imagemagick libxml2-dev libxslt1-dev openssl zlib1g libyaml-dev libxslt-dev autoconf libc6-dev ncurses-dev automake libtool bison libpq-dev libpq5 libeditline-dev
sudo apt-get install libreadline6 libreadline6-dev
- Install Git (on Linux rather than from Source)
- Install rvm
- Install Ruby
rvm install 1.9.3
rvm use 1.9.3 --default
Deploying the Application
Our example, Sentiment API, is located on GitHub. Try cloning the repository:
git clone email@example.com:jerzyn/api-demo.git
Now you can deploy the app by issuing:
And start the Thin server:
To access the API directly (without any security or access control) access:
You can find your-public-ip in the AWS EC2 Dashboard > Instances in the details window of your instance.
If you want to assign a custom domain to your Amazon instance, you’ll have to do one thing: Add an A record to the DNS record of your domain, mapping the domain to the public IP address. Your domain provider should either give you some way to set the A record (the IPv4 address), or it will give you a way to edit the nameservers of your domain. If they don’t allow you to set the A record directly, find a DNS management service, register your domain as a zone there, and the service will give you the nameservers to enter in the admin panel of your domain provider. You can then add the A record for the domain. Some possible DNS management services include ZoneEdit (basic, free) or Amazon route 53.
At this point, your API is open to the world. This is good and bad – great that you’re sharing, but bad that without rate limits, a few apps could kill the resources of your server and you would have no insight into who is using your API and how it’s being used. The solution is to add API management.
Enabling API Management with 3scale
Rather than reinvent the wheel and implement rate limits, access controls and analytics from scratch, we’ll leverage the 3scale API Management Platform. Get your free 3scale account, activate and log in to the new instance. The first time you log in, you can choose the option for some sample data to be created so you’ll have some API keys to use later. Next you’d probably like to go through the tour to get a glimpse on the system functionality (optional). Then start with the implementation.
To get some instant results, we’ll start with the API gateway in the staging environment which can be used while in development. Then we’ll also configure an Nginx proxy which can scale up for full production deployments. There’s some documentation on the configuration of the API proxy with 3scale, as well as more advanced configuration options.
Once you’ve signed in to your 3scale account, launch your API on the main Dashboard screen or Go to API > Select the service (API) > Integration in the sidebar and then choose Nginx on-premise proxy.
Set the address of of your API backend – this has to be the Public IP address unless the custom domain has been set, including http protocol and port 3000. Now you can save the changes to the API gateway in the staging environment to test your API by hitting the staging endpoint (after creating some app credentials in 3scale):
Where XXX is specific to your 3scale account and USER_KEY is the authentication key of one of the sample applications which you created when you first logged into your 3scale account. (If you missed that step just create a developer account and an application within that account).
Try it without app credentials, next with incorrect credentials, and then once authenticated, within and over any rate limits that you have defined. Only once it’s working to your satisfaction do you need to download the config files for Nginx.
Note: Any time you have errors, check whether you can access the API directly: your-public-dns:3000/v1/words/awesome.json. If this isn’t available, then you need to check whether the AWS instance is running and whether the Thin server is running on the instance.
Implement an Nginx Proxy for Access Control
In order to streamline this step we recommend that you install the fantastic OpenResty web application that is basically a bundle of the standard Nginx core with almost all the necessary 3rd party Nginx modules built-in.
sudo apt-get install libreadline-dev libncurses5-dev libpcre3-dev perl
Compile and install Nginx:
sudo wget http://agentzh.org/misc/nginx/ngx_openresty-220.127.116.11.tar.gz
sudo tar -zxvf ngx_openresty-18.104.22.168.tar.gz
./configure --prefix=/opt/openresty --with-luajit --with-http_iconv_module -j2
sudo make install
Now you can download the Nginx config files from the 3scale proxy page, and in the config file make the following changes:
- edit the .conf file from nginx download
- in line 28, which is preceded by info to change your server name put the correct domain (of your Elastic IP or custom domain name)
- in line 78 change the path to the .lua file, downloaded together with the .conf file.
We’re almost finished! The last step is to start the NGINX proxy and put some traffic through it. If it’s not running yet – remember the Thin server has to be started first – go to your EC2 instance terminal (the one you were connecting through ssh before) and start it now:
sudo /opt/openresty/nginx/sbin/nginx -p /opt/openresty/nginx/ -c /opt/openresty/nginx/conf/YOUR-CONFIG-FILE.conf
The last step will be verifying that the traffic goes through with a proper authorization. To do that, access:
where, APP_ID and APP_KEY are key and id of the application you want to access through the API call. Once everything is confirmed as working correctly, you’ll want to block public access to the API backend on port 3000, which bypasses any access controls.
If you encounter problems with the Nginx configuration or need a more detailed guide, I encourage you to check out our guide on configuring Nginx proxy. You can go completely wild with customization of your API gateway. If you want to dive more into the 3scale system configuration, like usage and monitoring of your API traffic, feel free to browse our Quickstart guides and HowTo’s.