Rackspace | DCMST

Bulk Delete Rackspace Cloud Files data via API

Wed, 13 Feb 2019 00:00:00 +0000

Sometimes it is necessary to delete all the content of the Cloud Files containers, however, the API does not have a proper method to delete the data and the containers on the same API call. Also, accoring to the documentation, you can only delete empty containers.

So, in cases where you need to delete the data and the containers at the same time, you should follow the next steps:

Download Turbolift, I know it is an old tool.

git clone https://github.com/cloudnull/turbolift
cd turbolift

In order to get and isolated installation, we are going to create a Python Virtual Environment (virtualenv)
```
mkvirtualenv turbolift
workon turbolift
```
Install the tool
```
pip install turbolift
```

Now, prior to start to play with the API calls, we need to grab some data to authenticate with the API:

Variable	Definition
USERNAME	This is the Rackspace Public Cloud username
APIKEY	This is your API-KEY
REGION	This is the Region where the Cloud Files are located (dfw, ord, iad, lon, hkg)
TOKEN	The TOKEN is generated after you get authenticated
ENDPOINT	This ENDPOINT is given also after you get authenticated

Next step, we are going to use cURL, to perform all the API calls:

First of all, get the TOKEN:

USERNAME=YOUR-USERNAME
APIKEY=YOUR-APIKEY
TOKEN=$(curl -s -XPOST https://identity.api.rackspacecloud.com/v2.0/tokens \
 -d'{"auth":{"RAX-KSKEY:apiKeyCredentials":{"username":"'$USERNAME'","apiKey":"'$APIKEY'"}}}' \
 -H"Content-type:application/json" | jq '.access.token.id' | tr -d "\"")

Next step, get the ENDPOINT:

ENDPOINT=$(curl -s -XPOST https://identity.api.rackspacecloud.com/v2.0/tokens \
 -d'{"auth":{"RAX-KSKEY:apiKeyCredentials":{"username":"'$CL_USERNAME'","apiKey":"'$APIKEY'"}}}' \
 -H"Content-type:application/json" | jq '.access.serviceCatalog[] | select((.name=="cloudFiles") or (.name=="cloudFilesCDN")) | {name} + .
 endpoints[] | .publicURL' | tr -d "\"" | grep -v cdn | grep -i $REGION)

In this case we are skipping all te CDN endpoints, but you can add them if is necessary.

With all the collected data, next step is use turbolift to delete the Cloud Files container and their data. To do it, I use a for-loop:

for i in $(curl -s -H "X-Auth-Token: $TOKEN" $ENDPOINT); do turbolift -u $USERNAME -a $APIKEY --os-rax-auth $REGION delete -c $i ; done

Now, you have all the Data and Cloud Files containers deleted on one region.

😄

WordPress with Let's Encrypt SSL Certificate on a Load Balancer

Sun, 03 Sep 2017 00:00:00 +0000

Hi again,

As many of you know a lot of “Production” applications need to be configured to provide High Availability. With that in mind, a best practice architecture to your application is to add a Load Balancer as a front end who distribute your traffic between your application nodes, as you can appreciate on the next image:

HA Load Balancer diagram

SSL Offloading

In this case, my “Production” application is my blog, and I will install a SSL Certificate on the Cloud Load Balancer(CLB) to offloading the encryption/decryption to the CLB instead of doing it on the webserver. That way your webservers uses port 80 (HTTP), as always, and you serve your content trought port 443(HTTPS).

SSL-Offloading diagram

Here are the what I use to configure my WordPress with SSL Certificate:

SSL Certificate issued using Let’s Encrypt
A Client of Let’s Encrypt called acme
A Cloud Load Balancer
A WordPress installation

Step 1: Install acme.sh client

There is a lot of ACME clients supported by Let’s Encrypt, the most popular is Certbot. However, I prefer to use acme.sh.

Let’s install it:

git clone https://github.com/Neilpang/acme.sh.git
cd acme.sh
# Create a data home directory
sudo mkdir -p /opt/acme/data
# Actual command to install it
bash acme.sh --install --home /opt/acme --config-home /opt/acme/data --certhome /opt/acme/data/ssl-certs --accountemail your@email.com

Step 2: Issue SSL Certificate

Once acme.sh is installed, we proceed to issue our first SSL Certificate:

/opt/acme/acme.sh --issue -d example.com -w /var/www/vhosts/example.com/public_html
[Mon Aug 25 06:04:07 UTC 2017] Creating domain key
[Mon Aug 25 06:04:07 UTC 2017] The domain key is here: /opt/acme/data/ssl-certs/example.com/example.com.key
[Mon Aug 25 06:04:07 UTC 2017] Single domain='example.com'
[Mon Aug 25 06:04:07 UTC 2017] Getting domain auth token for each domain
[Mon Aug 25 06:04:07 UTC 2017] Getting webroot for domain='example.com'
[Mon Aug 25 06:04:07 UTC 2017] Getting new-authz for domain='example.com'
[Mon Aug 25 06:04:08 UTC 2017] The new-authz request is ok.
[Mon Aug 25 06:04:08 UTC 2017] Verifying:example.com
[Mon Aug 25 06:04:11 UTC 2017] Success
[Mon Aug 25 06:04:11 UTC 2017] Verify finished, start to sign.
[Mon Aug 25 06:04:11 UTC 2017] Cert success.
-----BEGIN CERTIFICATE-----
MIIE/zCCA+egAwIBAgISA2AIs/G8gWjkRkNOUb7zmqh1MA0GCSqGSIb3DQEBCwUA
MEoxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MSMwIQYDVQQD
ExpMZXQncyBFbmNyeXB0IEF1dGhvcml0eSBYMzAeFw0xNzA4MjgwNTA0MDBaFw0x
NzExMjYwNTA0MDBaMBkxFzAVBgNVBAMTDmNvb2tpZWxhYnMubmV0MIIBIjANBgkq
hkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAo8/4fXH0dOHcSlyXpsBoULhwQYkz4m0J
MegRHU2mhyy/jfKWM6KHDxHpFWUFajLJ/ORE4uncvjmRYeSVBxgv2R2cYoZyKd6v
txT+Cdj3jD9fBfDerfdfsdfsd6Y6mlr6Im61afKsFXIgLsprBpK22JU6HOz+0Fdo
lan09aaF8zLPtVzdfJw9MU55K7nzerxO8j4ro2lve0PHExkMIBCrXey50wcuqQRY
hwkbbXsm+wTES7TCn3tooSzFq6ore3JrSckxhFQ96EOea0s9CgYnw4d9rU/b3jyK
bFCILEJK64vgFHx0qvd0hBJFJG/HUtAXAVrFQjjlZlCmCMbnee1UTQIDAQABo4IC
DjCCAgowDgYDVR0pasoasoasogWgMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEF
BQcDAjAMBgNVHRMBAf8EAjAAMB0GA1UdDgQWBBR2KRpXKKgTorwfXpo44wgKyFUl
QzAfBgNVHSMEGDAWgBSoSmpjBH3duubRObemRWXv86jsoTBvBggrBgEFBQcBAQRj
MGEwLgYIKwYBBQUHMAASdTdddHA6Ly9vY3NwLmludC14My5sZXRzZW5jcnlwdC5v
cmcwLwYIKwYBBQUHMAKGI2h0dHA6Ly9jZXJ0LmludC14My5sZXRzZW5jcnlwdC5v
cmcvMBkGA1UdEQQSMBCCDmNvb2tpZWxhYnMubmV0MIH+BgNVHSAEgfYwgfMwCAYG
Z4EMAQIBMIHmBgsrBgEEAYLfEwEBATCB1jAmBggrBgEFBQcCARYaaHR0cDovL2Nw
cy5sZXRzZW5jcnlwdC5vcmcwgasGCCsGAQUFBwICMIGeDIGbVGhpcyBDZXJ0aWZp
Y2F0ZSBtYXkgb25seSBiZSByZWxpZWQgdXBvbiBieSBSZWx5aW5nIFBhcnRpZXMg
YW5kIG9ubHkgaW4gYWNjb3JkYW5jZSB3aXRoIHRoZSBDZXJ0aWZpY2F0ZSBQb2xp
Y3kgZm91bmQgYXQgaHR0cHM6Ly9sZXRzZW5jcnlwdC5vcmcvcmVwb3NpdG9yeS8w
DQYJKoZIhvcNAQELBQADggEBAFVGs82tzyVER6U0x7p/Q+6xplDFd6ap/dVX9G6i
eRPf4ayGykPSH9J3ewu398LOQd3DE93oWbqc7PfEC40Z5HqvCEY3fl9auep99/IF
rwhf36J7PXvEsPrUB6pxNFSBw9WX366Z1MP8qoIzm3XYEpp2D/SPniWY5+eQ42Pj
WNxxVksA4kFUF9wgKcrsCNTm0X8GZj5HUXC1OwtlopY2w42QrAMGwz1jM4nxv5Mc
Jim+nT0zmJUhAdQi8ocDjAl2PvcfdgfmkMr9IWH3al/GJSKy3a9Cq+BaIsIUYi6E
8M8Mj+00ONNn1folm9aVn+FW5fVCaxYN32ir8PnoTWkOXK8=
-----END CERTIFICATE-----
[Mon Aug 25 06:04:11 UTC 2017] Your cert is in /opt/acme/data/ssl-certs/example.com/example.com.cer
[Mon Aug 25 06:04:11 UTC 2017] Your cert key is in /opt/acme/data/ssl-certs/example.com/example.com.key
[Mon Aug 25 06:04:11 UTC 2017] The intermediate CA cert is in /opt/acme/data/ssl-certs/example.com/ca.cer
[Mon Aug 25 06:04:11 UTC 2017] And the full chain certs is there: /opt/acme/data/ssl-certs/example.com/fullchain.cer

Where the explained options are:

-issue: Issue a new certificate

-d (-domain) : Specifies a domain, used to issue, renew or revoke, etc.

-w (-webroot) : Specifies the web root folder for web root mode. This is the DocumentRoot where your site is hosted and it is necessary to verify it by Let’s Encrypt.

Step 3: Install SSL Certificate on Cloud Load Balancer

So, at this moment we have our SSL Certificate, Private Key, and Intermediate CA Certificate ready to install on our Cloud Load Balancer (CLB)

Your cert is in /opt/acme/data/ssl-certs/example.com/example.com.cer
Your cert key is in /opt/acme/data/ssl-certs/example.com/example.com.key
The intermediate CA cert is in /opt/acme/data/ssl-certs/example.com/ca.cer

So we should go to https://login.rackspace.com -> Rackspace Cloud -> Networking -> Cloud Load Balancers:

Rackspace Portal - Cloud Loud Balancer

Then, to Optional Features and Enable/Configure on “Secure Traffic SSL”

Rackspace Portal - Cloud Loud Balancer

Finally, we add our SSL Certificate, Private Key, and Intermediate CA Certificate to the CLB and save the configuration:

Rackspace Portal - Cloud Loud Balancer

Step 4: Configure WordPress

We are almost done, at this time we already have configured our SSL on the CLB to provide WordPress over HTTPS, however, WordPress is still with HTTP, so we need to reconfigure our WordPress with SSL.

Database queries

First of all, we should update the links from http to https; we are going to do it directly on the database doing the following queries:

Change all instances of example.com to your own. If you have the www as part of your WordPress Address(URL) in the WordPress Settings, add the ‘www’.

Also, if you have a custom table prefix in the WordPress database, something other than the default ‘wp_’, then you must change all the instances of ‘wp_’ to your own table prefix.

Update any embedded attachments/img that use http:This one updates the src attributes that use double quotes:

UPDATE `wp_posts` SET post_content = REPLACE(post_content, 'src=\"http://example.com', \
'src=\"https://example.com') WHERE post_content LIKE '%src=\"http://example.com%';

This one takes care of any src attributes that use single quotes:

UPDATE `wp_posts` SET post_content = REPLACE(post_content, 'src=\'http://example.com', \
'src=\'https://example.com') WHERE post_content LIKE '%src=\'http://example.com%';

Update any hard-coded URLs for links.This one updates the URL for href attributes that use double quotes:

UPDATE `wp_posts` SET post_content = REPLACE(post_content, 'href=\"http://example.com', \
'href=\"https://example.com') WHERE post_content LIKE '%href=\"http://example.com%';

This one updates the URL for href attributes that use single quotes:

UPDATE `wp_posts` SET post_content = REPLACE(post_content, 'href=\'http://example.com', \
'href=\'https://example.com') WHERE post_content LIKE '%href=\'http://example.com%';

Update any “pinged” links:

UPDATE `wp_posts` SET pinged = REPLACE(pinged, 'http://example.com', \
'https://example.com') WHERE pinged LIKE '%http://example.com%';

This step is just a confirmation step to make sure that there are no remaining http URLs for your site in the wp_posts table, except the GUID URLs.

You must replace WP_DB_NAME, near the beginning of the query, with the name of your database.

This will confirm that nowhere in the wp_posts table is there a remaining http URL, outside of the GUID column. This ignores URLs in the GUID column.

This query only searches; it does not replace anything, nor make any changes. So, this is safe to run. It’s a safe and quick way to check the wp_posts table while ignoring the guid column.

This SQL query should return an empty set. That would mean that it found no http URLs for your site. (This is all just 1 query. It’s 1 very, very long line.)

Remember to replace WP_DB_NAME, near the beginning of the query, with the name of your database.

```sql
SELECT * FROM `WP_DB_NAME`.`wp_posts` WHERE (CONVERT(`ID` USING utf8) LIKE \
'%%http://example.com%%' OR CONVERT(`post_author` USING utf8) LIKE '%%http://example.com%%' \
OR CONVERT(`post_date` USING utf8) LIKE '%%http://example.com%%' \
OR CONVERT(`post_date_gmt` USING utf8) LIKE '%%http://example.com%%' \
OR CONVERT(`post_content` USING utf8) LIKE '%%http://example.com%%' \
OR CONVERT(`post_title` USING utf8) LIKE '%%http://example.com%%' \
OR CONVERT(`post_excerpt` USING utf8) LIKE '%%http://example.com%%' \
OR CONVERT(`post_status` USING utf8) LIKE '%%http://example.com%%' \
OR CONVERT(`comment_status` USING utf8) LIKE '%%http://example.com%%' \
OR CONVERT(`ping_status` USING utf8) LIKE '%%http://example.com%%' \
OR CONVERT(`post_password` USING utf8) LIKE '%%http://example.com%%' \
OR CONVERT(`post_name` USING utf8) LIKE '%%http://example.com%%' \
OR CONVERT(`to_ping` USING utf8) LIKE '%%http://example.com%%' \
OR CONVERT(`pinged` USING utf8) LIKE '%%http://example.com%%' \
OR CONVERT(`post_modified` USING utf8) LIKE '%%http://example.com%%' \
OR CONVERT(`post_modified_gmt` USING utf8) LIKE '%%http://example.com%%' \
OR CONVERT(`post_content_filtered` USING utf8) LIKE '%%http://example.com%%' \
OR CONVERT(`post_parent` USING utf8) LIKE '%%http://example.com%%' \
OR CONVERT(`menu_order` USING utf8) LIKE '%%http://example.com%%' \
OR CONVERT(`post_type` USING utf8) LIKE '%%http://example.com%%' \
OR CONVERT(`post_mime_type` USING utf8) LIKE '%%http://example.com%%' \
OR CONVERT(`comment_count` USING utf8) LIKE '%%http://example.com%%');
```

Now, we move to the wp_comments table. This changes any comment author URLs that point to the http version of your site. This is in case you’ve ever replied to a comment while your URL was pointing to http.
```
UPDATE `wp_comments` SET comment_author_url = REPLACE(comment_author_url, \
'http://example.com', 'https://example.com') WHERE comment_author_url \
LIKE '%http://example.com%';
```

This updates the content of the comments on your site. If there are any links in the comments that are linking to an http URL on your site, they will be updated to https.

UPDATE `wp_comments` SET comment_content = REPLACE(comment_content, 'http://example.com', \
'https://example.com') WHERE comment_content LIKE '%http://example.com%';

Now we move to the wp_postmeta table. This takes care of any custom post meta that points to the http version of your site.

UPDATE `wp_postmeta` SET `meta_value` = REPLACE(meta_value, 'http://example.com', \
'https://example.com') WHERE meta_value LIKE '%http://example.com%';

Now we move to the wp_options table. Update the WordPress Address (URL) and Site Address (URL).

For the WordPress Address URL, you may have to modify example.com. If you have WordPress installed in some other directory, then modify this according to your own WordPress URL. For example, some people have WordPress installed in a subdirectory named “blog”, and so their WordPress Address would be https://example.com/blog.
```
UPDATE `wp_options` SET `option_value` = "https://example.com" \
WHERE `wp_options`.`option_name` = 'siteurl';
```
This one will update the Site Address URL (this is the home page of your site):
```
UPDATE `wp_options` SET `option_value` = "https://example.com" \
WHERE `wp_options`.`option_name` = 'home';
```

WordPress Control Panel

Besides, with run the queries directly on the database, we can update, or verify, the blog URLs, by going to Settings > General

And updating your WordPress Address (URL) and Site Address (URL) address fields.

WordPress - Change URL

WordPress Config File

Finally, we should add the following line to our wp_config.php file

$_SERVER['HTTPS']='on';

Now, you have configured WordPress with Let’s Encrypt SSL Certificate on a Load Balancer.

Build a Dynamic DNS Client with Rackspace API

Mon, 11 Apr 2016 00:00:00 +0000

This time I’ve want to create a homemade Server with my Raspberry Pi2 and publish it using my own sub-domain, the main problem is that the ISP provide me an dynamic IP and we should ensure that if my IP address change the sub-domain record should point to the new IP.

The instructions assume that you:

Have a domain
Have already changed your NS records to point to dns1.stabletransit.com and dns2.stabletransit.com.

You should download the latest version of rsdns from github

cd ~/bin/
git clone https://github.com/linickx/rsdns.git

Go to your Rackspace portal https://login.rackspace.com/ and grab your Username & API key (It’s under “Your Account” -> “Account Settings” -> “API Key”)
Create a configuration file for rsdns (~/.rsdns_config) with your settings.
```
#!/bin/bash
RSUSER=dcmst
RSAPIKEY=1234567890
RSPATH=~/bin/rsdns/
```
You need your domain created on Rackspace(It’s under “Networking” -> “Cloud DNS” -> “Create Domain”) if you don’t have your domain created you are able to created using rsdns:
```
./rsdns-domain.sh -d www.d-cmst.io -e dcmst@devcrumbs.com
```
Once you have a domain setup you need to create an A record. To create the A record you going to need an IP address, you can use http://icanhazip.com to get your actual current IP. Again to create a record you are able to do it from Rackspace panel (It’s under “Networking” -> “Cloud DNS” -> YOUR_DOMAIN -> “Add Record”) or you can use rsdns:
```
./rsdns-a.sh -n dynamic-host.d-cmst.io -i 123.123.123.123 -t 3600
```
In the above the TTL is set to 1hr (3600 secs), this is so that DNS caches do not keep the record too long. That’s all the pre-work done, now lets get your dynamic host setup!
The script to update your a record is rsdns-dc.sh, and you run it like this:
```
./rsdns-dc.sh -n dynamic-host.d-cmst.io
```
The script uses icanhazip to get your current IP, it then update the A record with it.

I never switch off my router so I have create a created a cronjob to run that script every 2 hours, plus the 1hr TTL should mean that the record is roughly in sync with my IP without making unnecessary requests

I use CentOS, so I can simply drop the following file called rsdns-dc into /etc/cron.d/ with this…

vim /etc/cron.d/rsdns-dc
* */2 * * * dcmst /home/dcmst/bin/rsdns/rsdns-dc.sh -n dynamic-host.d-cmst.io &>/dev/null

Now we are done! Private Dynamic DNS on your own zone using the Rackspace API.

View sources IP's in Apache Logs behind a Load Balancer

Fri, 13 Feb 2015 00:00:00 +0000

When you use the Rackspace Cloud Load Balancers, it is common that the IP logged in Apache is the Private IP (ServiceNet) from the Cloud Load Balancer, however, we can fix that.

We can view sources IP’s in Apache Logs doing some changes on Apache configuration file and also on the vhosts configuration files.

On your Apache configuration file, you should to find the line:

LogFormat "%h %l %u %t \"%r\" %&gt;s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined

Modified to:

LogFormat "%{X-Forwarded-For}i %h %l %u %t \"%r\" %&gt;s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined

And also, on your vhosts configuration files you should to change the “combined” LogFormat definition will then be called in a “CustomLog” entry specific to your VirtualHost configuration. Here is an example VirtualHost definition to show you what I’m referring to:

ServerAdmin webmaster@example.com
DocumentRoot /var/www/html/example.com
ServerName example.com
ErrorLog logs/example.com-error_log
CustomLog logs/example.com-access_log combined

After adding the X-Forwarded-For definition to the LogFormat definition, you can restart Apache and view the logs to notice the difference. If all is done properly, you will see an actual public IP in the first field of your logs instead of the Cloud Load Balancer IP.

I'm a Racker

Mon, 31 Mar 2014 00:00:00 +0000

Since January 6th I working on Rackspace, the Open Cloud Company, so I’m a Racker almost 3 months ago and I’m loving every minute of it.

Rackspace - Kickoff

Everything stared on November 2013 when a Technical Recruiter contact me and started a proceess with some long tough interviews, ability tests, paperwork, etc; I have accepted a position as a Linux System Administrator I in the LATAM Team for Rackspace, so, I got to work for one of the most dynamic, fanatic, and fun tech companies in the world!

So, I was very excited, nervous, happy, all together.

The Castle

Since I came to The Castle, everything was wonderful, I have met nice, friendly, smart and fanatical people, like @SugarBear, a Rackspace Ambassador of Culture, or Graham Weston, Rackspace’s Chairman and Co-Funder. I met them at the Rookie Orientation (a.k.a Rookie’O), where I spend time with other Rookies learning about Rackspace history, culture and future plans.

On the Rookie’O, I was surprised and admired with all the energy that is transmitted between the new Rackers, it was awesome!

And I was inspired by the Rackspace Core Values:

Fanatical Support® in all we do.
Results first, substance over flash.
Committed to Greatness
Full Disclosure and Transparency
Passion for our Work
Treat fellow Rackers like Friends and Family

Which from my point of view I can applied to my personal life, and having great results.

Also is very comfortable to have a Coffe Shop, a soda machine or microwaves inside the Castle. It is pretty nice!

In general the first week in Rackspace, on the Rookie’O, I was a great experience, I can say that is one of my best experiences in my life.

Rackspace - Rookie O

Rackspace - Fanatical Jacket

Rackspace - Fuel station

Rackspace - Slide

Rackspace - Coffee Shop

My Goals

This is a new big challenge, because means:

Relocation in other country, specifically in San Antonio, TX, USA.
Leave my family in Mexico City, that means that I see my parents only for Skype or FaceTime :).
Know other culture, the “American” culture, with the Breakfast Tacos or Tex-Mex food (I really hate the Tex-Mex food yiack!) or the Lunch at noon, people do not always says “Good morning” and some details that I don’t understand but here is common.
Improve my skills in other language (English) event though I’m in the Rackspace LATAM team all the communications like emails or meetings are in English, so, it is very important for my job.
And the most important challenge for me is still learn about Linux, get my Red Hat Certifications, do my best at job and take advantage of this great opportunity. All of that to try to be a DevOps Engineer

Rackspace - LATAM Section

Rackspace - My Desk

I will be working on, providing Fanatical Support for our customers, resolving LATAM customer issues with Linux and working with remote teams from all around the world.

Summarizing, I’m a happy Racker 🙂

Rackspace | DCMST

Bulk Delete Rackspace Cloud Files data via API

WordPress with Let's Encrypt SSL Certificate on a Load Balancer

SSL Offloading

Step 1: Install acme.sh client

Step 2: Issue SSL Certificate

Step 3: Install SSL Certificate on Cloud Load Balancer

Step 4: Configure WordPress

Database queries

WordPress Control Panel

WordPress Config File

Build a Dynamic DNS Client with Rackspace API

View sources IP's in Apache Logs behind a Load Balancer

I'm a Racker

Previous

The Castle

My Goals