I am busy changing the world

In the previous blog post about setting logstash with ElasticSearch ( here ), I was output like this: input {         redis {                 host => "127.0.0.1"                 data_type => "list"                 key => "logstash"                 codec => json         } } output {         elasticsearch {                 embedded => true         } } As you can see, what logstash does is only move logs from redis to elasticsearch. There is no way it needs a lot of cpu power as I pointed out in the previous blog post. What's wrong? After a while searching around Google and digging the logstash documentation, I found out the reason. It is the " embbedded " option in my configuration. According to the logstash documentation about storing logs to elasticsearch: embedded Value type is boolean Default value is false Run the elasticsearch server embedded in this process . This option is useful if you wan

In the previous blog post , I had setup a log management system with logstash, elasticsearch, and redis. But I started logstash by running the java command for demonstration purpose. So, to run Logstash as a service in a Ubuntu machine: 1. Create a init script for logstash at /etc/init.d/logstash : (This will also start the Kibana web interface) This script is for logstash in the indexer server . For the shipper machine , change " mode ", " logstash_conf " and " command " definition: #! /bin/sh ### BEGIN INIT INFO # Provides:          logstash-indexer # Required-Start:    $remote_fs $syslog # Required-Stop:     $remote_fs $syslog # Default-Start:     2 3 4 5 # Default-Stop:      0 1 6 # Short-Description: Start daemon at boot time # Description:       Enable service provided by daemon. ### END INIT INFO . /lib/lsb/init-functions mode="indexer" name="logstash-$mode" logstash_bin="/usr/bin/java -- -jar /o

To set number of CPUs for a container (OpenVZ) in Proxmox VE: + Open the Proxmox console by click the Shell link in the web interface of proxmox: + In the opened console window, run vzlist to get CTID of containers: + Run the following command to set number of CPU for the container with CTID 102: # vzctl set 102 --cpus 2 --save References : http://pve.proxmox.com/wiki/Vzctl_manual

Deploying a Centralized Logs Management System seems very easy these days with such these great tools: + Logtash : collect logs, index logs, process logs, and ship logs + Redis : receive logs from logs shippers + ElasticSearch : store logs + Kibana : web interface with graphs, tables... We will implement the logs management system as the following architecture: (based on this article: http://michael.bouvy.net/blog/en/2013/11/19/collect-visualize-your-logs-logstash-elasticsearch-redis-kibana/ ) In  this tutorial, I only deploy one shipper (nginx logs of my Django app) on one machine, and one server to play as logs indexer (redis, logstash, elasticsearch, kibana): 1. On the indexer server, i nstall and run Redis :  http://iambusychangingtheworld.blogspot.com/2013/11/install-redis-and-run-as-service.html 2. O n the indexer server, i nstall and run ElasticSearch : $ sudo aptitude install openjdk-6-jre $ wget https://download.elasticsearch.org/elasticsearch/elast

To install and run Redis as a service in Ubuntu, following these steps: 1. Download and build Redis: $ wget http://download.redis.io/redis-stable.tar.gz $ tar xvzf redis-stable.tar.gz $ cd redis-stable $ make $ make test After the compilation the src directory inside the Redis distribution is populated with the different executables that are part of Redis: redis-server is the Redis Server itself. redis-cli is the command line interface utility to talk with Redis. redis-benchmark is used to check Redis performances. redis-check-aof and redis-check-dump are useful in the rare event of corrupted data files. It is a good idea to copy both the Redis server than the command line interface in proper places using the following commands: $ sudo cp redis-server /usr/local/bin/ $ sudo cp redis-cli /usr/local/bin/ 2. Install Redis: Create a directory where to store your Redis config files and your data: $ sudo mkdir /etc/redis $ sudo mkdir /var/redis Copy

This is the most basic and weakest encrypting method: Reverse Cipher def reverse_cipher(msg):     translated = ''     for i in reversed(range(len(msg))):         translated += msg[i]     return translated >>> reverse_cipher('This is so genius') 'suineg os si sihT'

" Effective Django " is a great tutorial made by Nathan Yergler which was presented at PyCon2013. All the texts : http://effectivedjango.com/ Video : https://www.youtube.com/watch?v=NfsJDPm0X54 This maybe the best thing to do this weekend. "Practice makes perfect", you know.

This is quite an interesting and common question in linux: List all the files in a folder that are owned by a the user "trinh", with permissions "-rw-r--r-- " 1. The easiest way is to use ls command and chain filter with grep : ls -lt . | grep trinh | grep '\-rw\-r\-\-r\-\-' 2. But, if you don't want to escape these permission characters, you can use awk to translate them into number. So, we will translate " -rw-r--r-- " as following: + "-" : the first character indicates whether it is a file or a directory + "rw-" : the next 3 characters is the owner's right, can be translated to 110 which means 6 in decimal system. + "r--": next 3 characters is the group's right, is 100 in binary and 4 in decimal. + "r--": the last 3 characters is the other users's right, 100 in binary, and 4 in decimal. So, "-rw-r--r--" is 644 in decimal system. ls -lt . | grep trinh | aw

I've just tried the we-all-know-what-it-is tool, git , to deploy and update changes to production server: 1. On my development computer, commit all changes and push to the repository: $ git add -A . $ git commit -m "Before deploying" $ git push origin master 2. On the production server: $ git clone myaccount@my.repo.address 3. In the development machine, update code, commit, and push to repository: $ git add -A . $ git commit -m "Updates" $ git push origin master 4. In production server, pull code from master: $ git pull origin master ...and so on It's quite interesting!

The easiest way to use Memcached as a cache backend for django is to enable it to cache the entire site: 1. Install Memcached : $ sudo apt-get install memcached The memcached service will run at 127.0.0.1:11211 by default. 2. Install python API for memcached: (myvirtualenv)$ pip install python-memcached 3. Enable Django to use Memcached by modified settings.py : MIDDLEWARE_CLASSES = (     'django.middleware.cache.UpdateCacheMiddleware', # should be the first line     'django.middleware.common.CommonMiddleware',     'django.contrib.sessions.middleware.SessionMiddleware',     'django.contrib.auth.middleware.AuthenticationMiddleware',     'django.middleware.csrf.CsrfViewMiddleware',     'django.contrib.messages.middleware.MessageMiddleware',     'django.middleware.cache.FetchFromCacheMiddleware', # should be the last line ) #memcache CACHES = {     'default': {         'BACKEND': 'django.core.cach

Just noted when I stopped the django-celery app which monitored by supervisord with: # supervisorctl stop myappcelery The status show myappcelery had stopped: # supervisorctl status myappcelery                         STOPPED    Nov 20 10:48 AM But, the python processed of django-celery were still running: # ps aux | grep celery root     25729  9.0  1.8 225676 38188 ?        S    11:07   0:00 python /mydjangoapp/manage.py celeryd --pool=eventlet -v 2 -B -s celery -E -l info root     25748  1.0  1.7 164640 36596 ?        S    11:07   0:00 python /mydjangoapp/ manage.py celeryd --pool=eventlet -v 2 -B -s celery -E Then, I take a look at the supervisord configuration for my django app: [program:ptccele] directory=/mydjangoapp/ command=/mydjangoapp/shellscripts/production/ptccele.sh user=root autostart=true autorestart=true stdout_logfile=/var/log/cele.log stderr_logfile= /var/log/ cele_error.log redirect_stderr=true I used a shellscript to run the celery w

Supervisord, a process control system, has a simple built-in web interface to help you manage processes. It is just so great!!! To enable it: 1. Add these line to /etc/supervisor/supervisord.conf to enable supervisord web interface in port 9001 (localhost, domain: my.domain.com): ... [inet_http_server] port=127.0.0.1:9001 username=guest password=mysecret ... +Restart supervisord: $ sudo supervisorctl reload 2. Pass requests of my.domain.com to 127.0.0.1:9001 : + Create a nginx configuration file for supervisord /etc/nginx/sites-available/supervisord : upstream supervisord { server localhost:9001 fail_timeout=0; } server {         listen 80;         server_name my.domain.com;         access_log /var/log/access_supervisor.log;         error_log /var/log/error_supervisor.log;         location / {                 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;                 proxy_set_header Host $http_host;                 proxy_re

There is one thing I want to take note about running a bash script with crontab: If I use the bash script alone, this will work: $ /path/to/my/script.sh #! /bin/bash export DISPLAY=:0 /path/to/my/pythonGUIapp.py & But if I run script.sh with crontab, It will not work: $ sudo crontab -u trinh -e 5 5  * * *  /path/to/my/script.sh To make it work, I have to change directory to the pythonGUIapp.py before executing it. So, change the script.sh as below: #! /bin/bash export DISPLAY=:0 cd /path/to/my/ ./pythonGUIapp.py &

Here are some great resources for readers or book lovers like me to read online: The Public Domain Review : http://publicdomainreview.org/ " The Public Domain Review is a not-for-profit project dedicated to showcasing the most interesting and unusual out-of-copyright works available online. All works eventually fall out of copyright – from classic works of art, music and literature, to abandoned drafts, tentative plans, and overlooked fragments. In doing so they enter the public domain, a vast commons of material that everyone is free to enjoy, share and build upon without restriction. We believe the public domain is an invaluable and indispensable good, which – like our natural environment and our physical heritage – deserves to be explicitly recognised, protected and appreciated. The Public Domain Review aims to help its readers to explore this rich terrain – like a small exhibition gallery at the entrance of an immense network of archives and storage rooms that lie

Today, I looked for a way to lock a Xubuntu machine remotely, and after a while of googling, I figured out this method: 1. I created 2 alias command to lock and unlock the monitor's screen (xscreensaver) in /home/myuser/.bash_aliases : alias lock='export DISPLAY=:0.0; xflock4;' alias unlock='export DISPLAY=:0.0; kill -9 "$(pidof xscreensaver)"; xscreensaver -no-splash &' 2. Install JuiceSSH (a ssh client) app in my Android phone: https://play.google.com/store/apps/details?id=com.sonelli.juicessh&hl=en 3. From my Android , connect to the Xubuntu machine via SSH with username ' myuser ', and then: To lock the screen, type this in my phone terminal: $ lock And unlock the screen with: $ unlock It's just fascinating! Feel like a hacker!!! \m/\m/\m/

Recently, whenever I open my computer's terminal, I always have to change directory to my projects folder. So, to avoid repeating that action, I add an alias to my Xubuntu's bash an alias: ~/.bash_aliases: alias projects='cd /var/projects/' Restart, and then I can only need to type ' projects ' to change my current directory to the project base. For more information: [0] http://www.ibm.com/developerworks/aix/library/au-getstartedbash/ [1] http://www.tldp.org/LDP/abs/abs-guide.pdf

This shell script I wrote in order to kill the python process if its CPU usage percentage exceeded 96%: #! /bin/bash THRESHOLD=96 while [ 1 ]; do     MYPROGRAM_ID=`pidof python`     # if the result is not empty, or there is a python process     if [ ! -z "$ MYPROGRAM _ID" ] ; then         CPU_LOAD=`ps -o %C -p $ MYPROGRAM _ID | tail -n 1`         echo $CPU_LOAD         # bc will return 0 for false and 1 for true         if [ $(echo "$CPU_LOAD > $THRESHOLD" | bc) -ne 0 ] ; then             kill -9 $GATEKEEPER_ID         else             echo "Normal"         fi     fi done Useful!

To get the CPU usage percentage of a specific process, run the following command: $ ps -o %C -p <PID> | tail -n 1 Example: to get CPU load of /usr/sbin/mysqld process which PID is 32241 : $ ps -o $C -p 32241 | tail -n 1 0.1

To add an autostart program in Xubuntu without using GUI tools: 1. Create a file in ~/.config/autostart/newautostartapp.desktop 2. Fill the file with: [Desktop Entry] Encoding=UTF-8 Version=0.9.4 Type=Application Name=New Autostart App Comment=New Autostart App Exec=<my application command> StartupNotify=false Terminal=false Hidden=false Restart and log-in again to see.

To disable right click for just desktop : xmodmap -e "pointer = 1 2 99" To apply for system wide: xmodmap -e 'pointer = 1 2 0 4 5 6 7 8 9' If you want to reverse it: xmodmap -e 'pointer = default' But, notice that it will restore to defaults upon rebooting.So, create an autostart script as following in ~/.config/autostart/disablerightclick.desktop : [Desktop Entry] Encoding=UTF-8 Version=0.9.4 Type=Application Name=Disable Right Click Comment=Disable Right Click Exec=xmodmap -e "pointer = 1 2 99" StartupNotify=false Terminal=false Hidden=false Restart and log-in again to see it work.

To set ENTER / RETURN key binding for a wx.Dialog, add the following to a button of that dialog: self.mybutton.SetDefault() For example: class LoginDialog (wx.Dialog):     def __init__ (self, parent, id=-1, title='Login',                  text='Please type your username and password:',                  username=''):         size = (400, 200)         wx.Dialog.__init__(self, parent, id, title, size=size)         self.panel = LoginPanel(self)         sizer = wx.GridSizer(1, 1)         sizer.Add(self.panel, flag=wx.ALIGN_CENTER | wx.ALIGN_CENTER_VERTICAL)         self.SetSizer(sizer)         self.Bind(wx.EVT_CLOSE, self.OnClose, self)         self.Show()     def OnClose (self, event):         self.Destroy() class LoginPanel (wx.Panel):     def __init__ (self, parent):         wx.Panel.__init__(self, parent)         self.setup_login_form()         self.do_layout()         self.setup_bindings()     def setup_login_form (self):         self.usernametxt = wx.

To launch a GUI program in a xUbuntu machine from a remote xUbuntu box, following these steps: 1. Assuming someone had logged-in to the remote machine (let call it R) with user ' remoteuser ' in graphical mode (init 5) 2. From my computer, ssh to R machine with user ' remoteuser ' (!important, if you login to R with another user, this won't work): trinh@local-pc $ ssh remoteuser@R.IP.Address 3. Run the following command before launch any graphical program: remoteuser@remote-pc $ export DISPLAY=:0 4. Finally, launch whatever GUI program you want, for example: Firefox: remoteuser@remote-pc$ firefox &

Early this morning, I installed a fresh Xubuntu 12.04 in a machine. After the computer booted up, the screen displays at 1024x768 mode. And I cannot changed to higher mode (actually, there was higher mode). So, I do the following steps to fix this issue: 1.Install this tool in a good machine (which is Xubuntu and does not have this problem) to get EDID (Extended Display Identification Data) of the monitor: $ sudo apt-get install read-edid 2. Run the following command to get the  monitor information (EDID): $ sudo get-edid | parse-edid The results will be something like:         # EDID version 1 revision 3 Section "Monitor"         # Block type: 2:0 3:ff         # Block type: 2:0 3:fc         Identifier "DELL E1912H"         VendorName "DEL"         ModelName "DELL E1912H"         # Block type: 2:0 3:ff         # Block type: 2:0 3:fc         # Block type: 2:0 3:fd         HorizSync 30-83         VertRefresh 56-75         # Max dot c

To allow only some specific users/groups access the SSH server: $ sudo nano /etc/ssh/sshd_config Add the following line to allow access only for user 'trinh': AllowUsers trinh Or this line to allow only group 'mygroup': AllowGroups mygroup Restart ssh service: $ sudo service ssh restart More options (source: http://knowledgelayer.softlayer.com/learning/how-do-i-permit-specific-users-ssh-access) : AllowGroups This keyword can be followed by a list of group name patterns, separated by spaces.If specified, login is allowed only for users whose primary group or supplementary group list matches one of the patterns.`*' and `?' can be used as wildcards in the patterns.Only group names are valid; a numerical group ID is not recognized.By default, login is allowed for all groups. AllowUsers This keyword can be followed by a list of user name patterns, separated by spaces.If specified, login is allowed only for user names that match one of the

Ubuntu 13.10 is the latest release when I post this blog. I tried to install MariaDB following these instructions: sudo apt-get install software-properties-common sudo apt-key adv --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 0xcbcb082a1bb943db sudo add-apt-repository 'deb http://ftp.yz.yamagata-u.ac.jp/pub/dbms/mariadb/repo/5.5/ubuntu saucy main' sudo apt-get update sudo apt-get install mariadb-server (source: https://downloads.mariadb.org/mariadb/repositories/) And I got this error message: The following packages have unmet dependencies: mariadb-server-5.5 : Depends: mariadb-client-5.5 (>= 5.5.33a+maria-1~saucy) but it is not going to be installed Depends: mariadb-server-core-5.5 (>= 5.5.33a+maria-1~saucy) but it is not going to be installed E: Unable to correct problems, you have held broken packages. The reason for this issue is the version of MariaDB is mismatch with the version in the Ubuntu repositories ( click to read more about this ).

The beauty of Python and any other OOP programing languages is the reusable functionality. So, I reuse the authentication module of Django to add a basic security layer for my wxPython application: + Authenticate a user with Django (django.contrib.auth.authenticate) + Save the session to a file names 'cred' + Read the 'cred' file to verify if that user has the privilege. * The main frame: class HomeFrame(wx.Frame):     def __init__(self):         wx.Frame.__init__(             self,             parent=None,             title='My wxPython application',             size=wx.DisplaySize(),         )         self.panel = HomePanel(self)         sizer = wx.GridSizer(1, 1)         sizer.Add(self.panel, flag=wx.ALIGN_CENTER)         self.SetSizer(sizer)         self.Bind(wx.EVT_CLOSE, self.OnClose, self)         self.Show()     def show_login_dialog(self):         login_dialog = LoginDialog(self)     def OnClose(self, event):         self.Destroy() class HomePa

I use this snippet to create a dialog for saving file using wxPython library: class ReportDialog(wx.Dialog):     def __init__(self, parent, id=-1, title='Report',                  text='Select Type Of Report'):         size = (350, 200)         wx.Dialog.__init__(self, parent, id, title, size=size)         self.panel = ReportPanel(self)         sizer = wx.GridSizer(1, 1)         sizer.Add(self.panel, flag=wx.ALIGN_CENTER | wx.ALIGN_CENTER_VERTICAL)         self.SetSizer(sizer)         self.Show() REPORT_TYPES = ['All', 'Teachers', 'Students', 'Parents'] class ReportPanel(wx.Panel):     def __init__(self, parent):         wx.Panel.__init__(self, parent)         self.setup_report_panel()         self.do_layout()         self.setup_bindings()     def setup_report_panel(self):         self.errmsg = wx.StaticText(self, label="")         self.errmsg.SetForegroundColour((255,0,0))         self.reporttypeselectbox = wx.ComboBox(self,