Do They Federate? #

See the overview, and also, RTC quick start.

Thursday, 16 May 2013

Strongbox and Aaron Swartz #
Building A Starship-Building Organization #

Ctein, writing at TOP:

Folks today may find this hard to believe, but at that time virtually all work on pollution problems was done by isolated specialists. Nobody even thought about the fact that studying something like air pollution axiomatically involved chemistry, biology, mechanical engineering, mathematical modeling, sociology, politics, economics, and meteorology…just to name a few relevant fields. Specialists got interested in some particular problem and studied it from the perspective of their specialty.

As an example, Dr. Clair Patterson was a geochemist who made the first accurate determination of the age of the earth by making extraordinarily sensitive measurements of lead isotopes in minerals. He was perpetually running into contamination problems in the lab, so he decided to track down the sources, which proved to be primarily leaded gasoline. He became the major figure in the fight to eliminate lead pollution.

Joe realized that multidisciplinary and diverse problems required a multidisciplinary and diverse intellectual culture to tackle them, in an era when “multidisciplinary” and “diverse” were barely notions. ARP reached out beyond the monolithic student body of Caltech to students from campuses around the country, who were invited to apply to work at ARP. Overwhelmingly, the ones ARP accepted were not white, male hard-science majors; Caltech had more than sufficient numbers of those.

EFF: Opening a New Front Against Secret IP Treaties #

Monday, 06 May 2013

Zoom Lenses #

Mike Johnston:

The point I want to make here is simply that the advent of the “zoom norm” simply meant that camera users in general tended to become more and more ignorant of the meaning of focal lengths and angles of view. The camera came with a lens; it was “wide” at one end and “tele” at the other; and there you had it.

Sunday, 05 May 2013

     __|__
-------O-------
    o´   `o

Textdrive Server Deployment with Nginx via FastCGI

My notes on deploying a Django application (this website) on Textdrive shared hosting with Nginx via FastCGI.

Thanks to Evan Carmi, who much of this is paraphrased from, and Pokoka who has far more knowledge on system administration than I, and has an alternate method using uWSGI and a handy script.

Search and Replace

yourusername
yourdomain.tld
yourproject
yourportnumber

Server folder layout

yourusername
    domains
        yourdomain.tld
            .virtualenv     <- virtual environment
                yourproject
                    bin
                    include
                    lib
                        python2.7/site-packages     <- where django and packages installed with pip go
                    src
            etc
                nginx
                    sites-enabled
                        nginx.conf    <- nginx config file (=todo:create symlink from here to web/yourproject/yourproject/conf below)
            web
                yourproject
                    yourproject     <- django project git repository cloned from github
                    init.sh     <- fcgi startup script. don't forget to make executable with chmod +x init.sh
                    manage.py
            public
                static   <- static files - either create symlink from here to project repo or run python manage.py collectstatic
                static/files  <- static files uploaded by users of your app
                static/@admin  <- django admin media symlink

SSH into your Txd server

$ ssh yourusername@yourserver.textdrive.us

Install virtualenv

$ mkdir -p local/
$ cd local/
$ wget -O virtualenv-1.9.1.tar.gz http://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.9.1.tar.gz
$ tar xzvf virtualenv-1.9.1.tar.gz

Let’s now create our virtualenv on our server. This is the same as on our local machine:

$ python virtualenv.py --no-site-packages ~/domains/yourdomain.tld/.virtualenv/yourproject
$ cd ~/domains/yourdomain.tld/.virtualenv/yourproject
$ . bin/activate

Setup the server’s virtualenv

Create a symbolic link called ‘yourproject’ in ~/domains/yourdomain.tld/.virtualenv/yourproject/lib/python2.7/site-packages:

$ ln -s `pwd` ../lib/python2.7/site-packages/`basename \`pwd\`` 
$ export DJANGO_SETTINGS_MODULE=yourproject.settings

Add the previous line - export DJANGO_SETTINGS_MODULE=yourproject.settings - to the ~/domains/yourdomain.tld/.virtualenv/yourproject/bin/activate file:

$ echo "!!" >> ../bin/activate

Activate virtualenv

$ source ~/domains/yourdomain.tld/.virtualenv/yourproject/bin/activate

Install Django, then upload your application to the server

$ pip install django
$ cd ~/domains/yourdomain.tld/web/
$ python django-admin.py startproject yourproject

This will have created the following folder layout under ~/domains/yourdomain.tld/web/:

yourproject
    manage.py
    yourproject     <- default django app

Remove the latter yourproject folder (the default django app created by the startproject command) and replace it with your own project cloned from Github or elsewhere, since, like a good citizien, you are developing locally and only putting tested applications on the production server. Leave manage.py alone.

$ cd ~/domains/yourdomain.tld/web/yourproject/
$ rm -rf yourproject
$ git clone https://github.com/yourusername/yourproject/ yourproject
Initialized empty Git repository in /users/home/yourusername/domains/yourusername/web/yourproject

Set up your settings files for production (=TODO: Set up django-configurations) and update the database settings to use your PostgreSQL or MySQL database if you are using those instead of sqlite3. If you use PostgreSQL or MySQL you need to create those first in virtualmin.

DATABASES = { 
    'default': { 
        'ENGINE': 'postgresql_psycopg2', 
        'NAME': 'yourusername_django_mysite_database', 
        'USER': 'yourusername', 
        'PASSWORD': 'password', 
        'HOST': 'localhost', 
        'PORT': '5432', 
    } 
}

Try running python manage.py syncdb. If it works then your database is configured correctly.

Install required software packages on the server with pip

Let’s install the software packages from your application’s requirements.txt file (if you have one).

$ cd ~/domains/yourdomain.tld/web/yourproject/yourproject
$ pip install -r requirements.txt

Setup static media

Let’s assume you have some static media for your project in ~/domains/yourdomain.tld/web/yourproject/yourproject/static

For security reasons (but don’t trust me on this) we don’t want to serve static media (CSS, JavaScript, images) from inside our project directory. Instead, let’s create some other directories to serve static media from:

$ mkdir -p ~/domains/yourdomain.tld/web/public/static

And then create a symbolic link from there to our media directory.

$ ln -s ~/domains/yourdomain.tld/web/yourproject/yourproject/static/ ~/domains/yourdomain.tld/web/public/static

Now let’s link Django’s contrib.admin media to this location so that the static assets of Django’s admin app get served as well:

$ ln -s ~/domains/yourdomain.tld/.virtualenv/yourproject/lib/python2.7/site-packages/django/contrib/admin/static/admin/ ~/domains/yourdomain.tld/web/yourproject/yourproject/static/admin

And lastly let’s configure settings.py (or settings_local.py depending on your project setup) to use these locations:

import os
import sys

from os.path import dirname, join
from sys import path

path.append(join(dirname(__file__), "yourproject"))
PROJECT_ROOT = os.path.abspath(os.path.dirname(__file__))

STATIC_URL = 'http://domain.tld/static/'
ADMIN_MEDIA_PREFIX = os.path.join(PROJECT_ROOT, "admin/")

Setup NginX and FastCGI

Create a directory to keep your nginx.conf file and the nginx.conf file itself:

$ mkdir -p ~/yourdomain.tld/etc/nginx/sites-enabled 
$ vim ~/yourdomain.tld/etc/nginx/sites-enabled/nginx.conf

=TODO: keep nginx conf in the project’s git repository and create symlink from sites-enabled to yourproject/conf/nginx.conf. This way we can keep the config in version control.

Edit your nginx.conf file to look like the following but with your own port number1 (yourportnumber), domain (yourdomain.tld), and username (yourusername).

# http://stackoverflow.com/questions/13371925/how-to-turn-off-or-specify-the-nginx-error-log-location
error_log /dev/null crit;

worker_processes 1;
pid /users/home/yourusername/domains/yourdomain.tld/tmp/nginx.pid;

events { 
    worker_connections24; 
}

http { 
    include     /opt/local/etc/nginx/mime.types;

    client_body_temp_path /users/home/yourusername/domains/yourdomain.tld/var/spool/nginx/client_temp 1 2;
    proxy_temp_path /users/home/yourusername/domains/yourdomain.tld/var/spool/nginx/proxy_temp 1 2;
    fastcgi_temp_path /users/home/yourusername/domains/yourdomain.tld/var/spool/nginx/fstcgi_temp 1 2; 
    uwsgi_temp_path /users/home/yourusername/domains/yourdomain.tld/var/spool/nginx/uwsgi_temp 1 2; 
    scgi_temp_path /users/home/yourusername/domains/yourdomain.tld/var/spool/nginx/scgi_temp 1 2;

    log_format main '$remote_addr - $remote_user [$time_local] '
                  '"$request" $status$body_bytes_sent "$http_referer" '
                  '"$http_user_agent" "$http_x_forwarded_for"';
            
    access_log /users/home/yourusername/domains/yourdomain.tld/logs/nginx/nginx.access.log;
    error_log /users/home/yourusername/domains/yourdomain.tld/logs/nginx/nginx.error.log;

    server { 
        listen       yourportnumber; 
        server_nameyourdomain.tld; 
        # error_log /users/home/yourusername/domains/yourdomain.tld/logs/nginx-fcgi.error.log;

        location /static/files{
            alias /users/home/yourusername/domains/yourdomain.tld/web/public/static/files;
        }

        location /static {
            alias /users/home/yourusername/domains/yourdomain.tld/web/public/static;
        }

        location / { 
            fastcgi_pass unix:/users/home/yourusername/domains/yourdomain.tld/web/yourproject/yourproject.socket;

            # fastcgi parameters 
            fastcgi_param PATH_INFO $fastcgi_script_name; 
            fastcgi_param QUERY_STRING $query_string; 
            fastcgi_param REQUEST_METHOD $request_method; 
            fastcgi_param SERVER_PORT $server_port; 
            fastcgi_param SERVER_PROTOCOL $server_protocol; 
            fastcgi_param SERVER_NAME $server_name; 
            fastcgi_param CONTENT_TYPE $content_type; 
            fastcgi_param CONTENT_LENGTH $content_length; 
        }  
    } 
}

Create an init.sh script in your project directory to start the Django FastCGI process that should look like:

#!/usr/local/bin/bash

#Activate the virtualenv 
source /users/home/yourusername/domains/yourdomain.tld/.virtualenv/yourproject/bin/activate

PROJECT_NAME="yourproject"
PROJECT_DIR="/users/home/yourusername/domains/yourdomain.tld/web/yourproject/yourproject" 
PID_FILE="/users/home/yourusername/domains/yourdomain.tld/web/yourproject/yourproject.pid" 
SOCKET_FILE="/users/home/yourusername/domains/yourdomain.tld/web/yourproject/yourproject.socket" 
BIN_PYTHON="/users/home/yourusername/domains/yourdomain.tld/.virtualenv/yourproject/bin/python" 
# DJANGO_ADMIN="/users/home/yourusername/domains/yourdomain.tld/.virtualenv/yourproject/bin/django-admin.py" 
MANAGE="/users/home/yourusername/domains/yourdomain.tld/web/yourproject/manage.py" 
OPTIONS="maxchildren=2 maxspare=2 minspare=1"
METHOD="prefork"

case "$1" in
    start) 
      # Starts the Django process 
      echo "Starting Django project" 
      # $BIN_PYTHON $DJANGO_ADMIN runfcgi $OPTIONS method=$METHOD socket=$SOCKET_FILE pidfile=$PID_FILE 
      $BIN_PYTHON $MANAGE runfcgi $OPTIONS method=$METHOD socket=$SOCKET_FILE pidfile=$PID_FILE 
  ;;
    stop) 
      # stops the daemon by cating the pidfile 
      echo "Stopping Django project" 
      kill `/bin/cat $PID_FILE` 
  ;;
    restart) 
      ## Stop the service regardless of whether it was 
      ## running or not, start it again. 
      echo "Restarting process" 
      $0 stop
      $0 start
  ;;
    *)
      echo "Usage: init.sh (start|stop|restart)" 
      exit 1
  ;;
esac

Make this init.sh file executable:

$ chmod +x /users/home/yourusername/domains/yourdomain.tld/web/yourproject/init.sh

Start Django FastCGI instance with:

$ /users/home/yourusername/domains/yourdomain.tld/web/yourproject/init.sh start

This script also takes stop, and restart as parameters.

Launch Nginx with your configuration file:

$ /usr/local/sbin/nginx -p /users/home/yourusername/ -c /users/home/yourusername/domains/yourdomain.tld/etc/nginx/sites-enabled/nginx.conf

The Django application should now be running at http://domain.tld:PORTNUBMER/. Don’t forget to log in and go to http://domain.tld:yourportnumber/admin/sites/site/ and set the domain name.

Create a ProxyPath and ProxyPathReverse from http://domain.tld:PORTNUMBER to http://domain.tld

  1. Log into your txd account through virtualmin (https://virtualmin-yourserverlocationid.textdrive.us/yourserver/)
  2. Under “Server Configuration” click on “Proxy Paths”
  3. Click “Add a new proxy path.”
  4. Enter “/” for the “Local URL path”
  5. Enter “http://127.0.0.1:yourportnumber” for the “Destination URL”.
  6. Click “Save”

ProxyPassReverse:

  1. Click “Services”
  2. Click “Configure website”
  3. Click “Show Directives”
  4. From the drop down list chose “ProxyPassReverse”
  5. Find the heading “Map remote Location: headers to local”
  6. For “Local URL Path” add ”/”
  7. For “Remote URL” add “balancer://root/”

NOTE: the balancer name should be the same as the one entered for “Remote URL” under “Map local to remote URLs”.

=TODO: create bootup actions in virtualmin to start the Django FastCGI process and NginX on server reboots.

Stopping Nginx

$ ps -ef | grep nginx | awk '{print $2}'| xargs kill -9

Stopping fcgi (whenever you update your application you need to stop and then start fcgi)

$ . init.sh stop

Todo:

Set up fabric script to automate deployment.

  1. To find a free port number you can use for your account, log into your Textdrive account via virtualmin, then click Other Tools and Check Ports. ↩

     __|__
-------O-------
    o´   `o

Friday, 26 April 2013

Unified Login in Django and Drupal #

This is going to be useful at work, where, unfortunately, I also need to deal with Drupal.

Induction #

Polyglot database client for OS X.

Wednesday, 24 April 2013

BitTorrent Sync #

Sync your files without 3rd party servers involved. Bye-bye Dropbox.

Saturday, 20 April 2013

Verlet-js #

Verlet integration physics engine written in JavaScript.

Friday, 19 April 2013

Transcript of Secret Meeting Between Julian Assange and Google CEO Eric Schmidt #

Wednesday, 17 April 2013

Paper.js #

Open source vector graphics scripting framework that runs on top of the HTML5 Canvas.

Textmate Missing Drawer #

Useful.

Monday, 15 April 2013

The Rise of the New Groupthink #

Saturday, 13 April 2013

What Do You Think of Indians? #

Questioner: What do you think of Indians?

Krishnamurti: That is really an innocent question, is it not? To see facts without opinion is one thing, but to have opinions about facts is totally another. It is one thing just to see the fact that a whole people are caught in superstition, but quite another to see that fact and condemn it. Opinions are not important, because I will have one opinion, you will have another, and a third person will have still another. To be concerned with opinions is a stupid form of thinking. What is important is to see facts as they are without opinion, without judging, without comparing.

To feel beauty without opinion is the only real perception of beauty. Similarly, if you can see the people of India just as they are, see them very clearly without fixed opinions, without judging, then what you see will be real.

The Indians have certain manners, certain customs of their own, but fundamentally they are like any other people. They get bored, they are cruel, they are afraid, they revolt within the prison of society, just as people do everywhere else. Like the Americans, they also want comfort, only at present they do not have it to the same extent. They have a heavy tradition about renouncing the world and trying to be saintly; but they also have deep-rooted ambitions, hypocrisy, greed, envy, and they are broken up by castes, as human beings are everywhere else, only here it is much more brutal. Here in India you can see more closely the whole phenomenon of what is happening in the world. We want to be loved, but we don’t know what love is; we are unhappy, thirsting for something real, and we turn to books, to the Upanishads, the Gita, or the Bible, so we get lost in words, in speculations. Whether it is here, or in Russia, or in America, the human mind is similar, only it expresses itself in different ways under different skies and different governments.

From Think on These Things (PDF), Chapter 11: Conformity and Revolt, by Jiddu Krishnamurti.

News is Bad for You #

Via Tony ‘Montana’ Jones.

Friday, 12 April 2013

Hack You Maps #

Wednesday, 10 April 2013

Thug Kitchen #

Thanks, BB.

Monday, 08 April 2013

How to Share Your Data Effectively #

See also: Heart of Nerd Darkness: Why Updating Dollars for Docs Was so Difficult or, PDF hell.

Front-end Performance for Web Designers and Front-end Developers #

Friday, 05 April 2013

Negative Capability #

John Keats used the term negative capability to describe the artist’s receptiveness to the world and its natural marvel, and to reject those who tried to formulate theories or categorical knowledge. In this concept, Keats posited the world and the human to be of infinite depth. Such a position put Keats at the forefront of the Romantic movement, and even at the cusp of modernism, according to some commentators

In a letter to his brothers, George and Thomas Keats, on December 21, 1817, Keats used the phrase negative capability for the first and only time. He did so in criticism of Coleridge, who he thought sought knowledge over beauty:

I had not a dispute but a disquisition with Dilke, upon various subjects; several things dove-tailed in my mind, and at once it struck me what quality went to form a Man of Achievement, especially in Literature, and which Shakespeare possessed so enormously – I mean Negative Capability, that is, when a man is capable of being in uncertainties, mysteries, doubts, without any irritable reaching after fact and reason – Coleridge, for instance, would let go by a fine isolated verisimilitude caught from the Penetralium of mystery, from being incapable of remaining content with half-knowledge. This pursued through volumes would perhaps take us no further than this, that with a great poet the sense of Beauty overcomes every other consideration, or rather obliterates all consideration.

Keats understood Coleridge as searching for a single, higher-order truth or solution to the mysteries of the natural world. He went on to find the same fault in Dilke and Wordsworth. All these poets, he claimed, lacked objectivity and universality in their view of the human condition and the natural world. In each case, Keats found a mind which was a narrow private path, not a “thoroughfare for all thoughts.” Lacking for Keats were the central and indispensable qualities requisite for flexibility and openness to the world, or what he referred to as negative capability.

Thursday, 04 April 2013

5 Ways to Implement HTTPS in an Insufficient Manner (And Leak Sensitive Data) #

Use HTTPS everywhere, and make sure you set the HSTS header.

Vintage Covers from NASA Space Program #

Monday, 01 April 2013

Building Anti-Fragile Architecture at Netflix #

From one of the slides:

Google’s software architecture arises from two basic insights. First, we provide reliability in software rather than in server-class hardware, so we can use commodity PCs to build a high-end computing cluster at a low-end price…

— Web Search for a Planet: The Google Cluster Architecture

Check out NetflixOSS here. Thanks, Bryan.

Thursday, 28 March 2013

Material Honesty #

Don’t make something look like something it’s not.

Paul Robert Lloyd’s article, “The Web Aesthetic,” lays the foundation.
The web could almost be considered a composite, made up of HTTP (the how), URLs (the where), and HTML (the what). Omit any one of these ingredients and you’re no longer building the web.

i.e. Don’t use Adobe Flash.

Wednesday, 27 March 2013

The DDoS That Almost Broke the Internet #
Forecast #

Excellent weather service.

Monday, 25 March 2013

Which Loads Faster? #

Thanks, Peter.

Sunday, 24 March 2013

Why Use Nginx #
Flask Tutorial #

Complete.

About the Logbook

The Hypertexthero Logbook is a blog about web, design and simplicity. It includes links to other places on the internet that Hypertexthero finds interesting.