Monday, June 29, 2015

A bash script to backup web application files and database to Dropbox | ineed.coffee

A bash script to backup web application files and database to Dropbox | ineed.coffee



The Problem

How much does it take you to write a blog post? What about 100 posts? What about writing the blog engine itself?
What if you suddenly loose everything? What if it was not your fault and your hosting provider does not backup your data? This is an extreme (but not unlikely) case.
Yesterday, I was greeted by an error page from all of my websites.
The problem was due to MySQL related issues. Fortunately, Webfaction is run by clever developers who solved the issue in a matter of minutes. They backup data, as well. However, hosting providers are not required to do backups of your data on a hourly basis, not even on a daily basis.
This made me think. Since the infamous black day where I (stupidly) lost 6 years of data, I occasionally do a manual backup of my web applications and store them on my computer and external storage. What about a shell script that automatically backups all of our web application’s files and databases, and optionally uploads everything to DropBox?

The Solution

Here is my solution:

Download

You can download the script here.

Configuration

The script assumes that all the websites and web applications live under the same directory, e.g., /home/dgraziotin/webapps.
It also assumes that each web application can have up to one MySQL (or PostgreSQL) database and a database user with full access on it. On Webfaction, each new database is accompanied by the creation of a user with the same name of the database. Only this user can access to the database. This is an elegant (and secure) solution, which should be employed on any other hosting platform and server.
We store all the information in the file /home/yourusername/.backup_passwd.
Below is an example of the password file:
The provided password file contains three entries.
The first one refers to the website myblog, using a MySQL database calleddgraziotin_myblog. The user dgraziotin_myblog accesses it with the passwordsaD23fWffFEfwg.
The second entry refers to a website called landing_page, not using any database at all.
The third entry is for a website called myecommercesite. It uses a PostgreSQL databas,dgraziotin_esite. The user dgraziotin_esite owns it with the password DFWkf3kf24KGKf.
Regarding Dropbox integration, install first Dropbox-Uploader and uncomment the last four lines of the script. Adjust the path of dropbox_uploader.sh to where you put the script.

How does it work?

The script exploits the Internal Field Separator of GNU/Linux to define the separator to identify patterns of text.
At line 36, we set the IPS to \n, or newline. Because of this, the for loop at line 38 will iterate every line of the password file.
Within the for loop, we set IPS to " ", or space character.
Each line is split up to three times, according to the number of space character encountered. These three substrings are stored in the variables APP_NAMEDB_NAME, andDB_USER.
These values are used in lines 49-51 to create a bzipped archive of the website (or the web application).
In lines 53-63 we create a bzipped archive of the database (MySQL or PostgreSQL), if the database has been defined in the password file.
I am using it through a cronjob to backup all of my websites. I hope that someone else finds it useful, too. Feel free to comment on this post to report possible bugs.

Friday, June 26, 2015

How to inspect iOS's HTTP traffic without spending a dime

How to inspect iOS's HTTP traffic without spending a dime



I recently had a problem. I was seeing intermittent issues with an iPhone app, Tapatalk, not working properly with a web forum hosted by a friend of mine. I knew there was a much better chance of getting the bug fixed if I could a) prove it was a bug and b) show the devs exactly where the problem was, but I was hampered by the usual problem: iOS apps are a bit of a black box, and I couldn't see what it was doing internally.
However, like almost all network-aware iOS apps, this one was clearly using a web service to get data from the backend. So, all I needed to do was figure out a way to see the traffic on the web service. This is the sort of thing I used to do all the time when my day job was writing load testing scripts for big ecommerce sites, but the first time I'd had to do it on the Mac or from an iOS client. I managed to get it working after doing a little research. If you find yourself in need of a similar solution -- perhaps for iOS app development, reporting a bug or some other reason (or just plain hacker interest!) -- then click through for step-by-step instructions on how to intercept and view your iOS web traffic from any Mac running on the same network.
The first thing you need is an HTTP sniffer program. The grandaddy of all network traffic sniffers is Wireshark, but it's rather low-level and overpowered for quickly looking through HTTP traces. It's a bit like using an electron microscope when what you wanted was a magnifying glass. I came across several glowing references to Tuffcode's MacScoop HTTP Scoop during my research, but didn't really want to spend $15 on an app I was only going to use once. I settled on PortSwigger's Burp Suite, a comprehensive HTTP security analysis tool. The free version has the HTTP Proxy feature, which is the only bit we need; grab that. (Windows users: you can use the excellent and free Fiddler Web Debugger, but I won't be walking you through that today, sorry. The steps are very similar though.)
Burp Suite is a Java program, so when it downloads, you'll see a directory with a JAR file in it. If you double click that, it should start up after a warning about an untested JVM version. Be thankful we still get JVMs with our OS X for now or this would be more complex.
Burp is written by and for security experts, so the UI is a bit ... Spartan, ... but it's easy to configure it for the simple feature we want to do. First, click the Proxy tab at the top, then click the "intercept is on" button to make it say "intercept is off," like so:
Next, select the Options tab. It'll show you a single "proxy listener" running in a list. Click on it, then click the Edit button. Untick the "Listen on loopback interfaces only" checkbox, then click "Update." You'll get a warning you can ignore, and when you're done, the app window should look like this:
When you're done, click on the History tab. If you're one of the many Mac users who find Java UIs give you hives, you can rejoice, because you're done with that for now!
Next, open Apple > System Preferences > Network. Select your current active network connection and make a note of the IP address your Mac is using. For me, this is 192.168.2.32.
Now, turn to your iOS device -- I'll be using my iPhone, but this works the same way on an iPad or an iPod touch. Go into Settings > Wi-Fi > your Wi-Fi network, and then click the blue "more details" arrow. Scroll down and at the bottom there's a set of three buttons under "HTTP Proxy." Select "Manual" and fill in your Mac's IP address (found in the last step) and the Burp Suite port number (8080 unless you changed it earlier), like so:
And that's it. If you flick back to the Burp Suite window on the Mac now and start browsing the Web or using apps on your iPhone, you'll see all the traffic show up in a neat little list. You can click on each individual request to see the full text sent and received as part of the request. Note that you might see some security warnings on the iPhone for any app that uses an encrypted link; this is because the app thinks that Burp Suite might be a man-in-the-middle attack. It's not, though, and it's safe to ignore that warning.
And there you have it -- with one free download and a few minutes of configuration work, you can snoop all of iOS's web traffic for fun and/or profit.
Update: corrected name "MacScoop" to "HTTP Scoop". Thanks for the correction, Jonathan.
Update 2: please check the comments for lots of suggestions for alternate tools to do this job. Thank you all for your insightful additions!

Monday, June 22, 2015

How to find outdated joomla versions on your server to reduce the risk of being hacked

How to find outdated joomla versions on your server to reduce the risk of being hacked



Today I want to focus on a topic that can lead to huge problems of hacked accounts, spam mailings etc.: Outdated Joomla installations on your server.
Of course, this is valid for other software, too. The mentioned method should work in a similar way for other software. Since Joomla is widely spread throughout the internet, especially on shared hosting systems CMS (Content Management System), this howto will only cover Joomla so far.

Prerequisites

You will need at least the tools bc and sed that are not always installed by default.

Let us begin

At first there is one question: how can I recognize the joomla version from the installed files?
This depends on the installed version of Joomla. Until now I have found three different files that contain the version information:
/libraries/joomla/version.php
/libraries/cms/version/version.php
/includes/version.php
The most important lines of these files are the version number and the minor version that are contained in the following variables:
var $RELEASE = '1.0';
var $DEV_LEVEL = '12';
In a next step we search for the most recent version of Joomla on the official website. At the time of writing this howto there are three version groups:1.5 (1.5.26), 2.5 (2.5.17) and 3.2 (3.2.1).

Find joomla installations on your server

One thing that all joomla installations have in common is the folder name "components", so we search for all folders with this name. At the same time we ignore all those components folders that are contained in a subfolder "administrator". The base path /var/www has to be adjusted if the websites on your server do not lie in there.
find /var/www/ -type d -name 'components' ! -wholename '**/administrator/components'
This command will give you a list of all those folders. The command dirname is suitable for getting the path that contains the components folder. This is the base path of the Joomla installation.
We do this in a loop for all found components folders:
for L in `find /var/www/ -type d -name 'components' ! -wholename '**/administrator/components'` ; do
    D=`dirname $L` ;
done

Get the major and minor version

To get the version of your installation we will use the combined commands "grep" and "sed".
First we check which of the three files I mentioned earlier exists in the installation path.
F=$D/libraries/joomla/version.php ;
F2=$D/libraries/cms/version/version.php ;
F3=$D/includes/version.php ;
if [[ -e "$F" || -e "$F2" || -e "$F3" ]] ; then
    if [[ -e "$F" ]] ; then
        F=$F ;
    elif [[ -e "$F2" ]] ; then
        F=$F2 ;
    elif [[ -e "$F3" ]] ; then
        F=$F3 ;
    fi
else
    echo "No joomla version file found." ;
fi
Now we read the major and minor version from this file:
VERSION=`grep '$RELEASE' $F | sed -r "s/^.*=\s*'(.*)'.*$/\1/g"` ;
SUBVERSION=`grep '$DEV_LEVEL' $F | sed -r "s/^.*=\s*'(.*)'.*$/\1/g"` ;

Compare versions

Since version numbers are no integers, we cannot compare them inside the bash script using -lt etc. Instead we need to use an external programm called bc:
ISOK=1 ;
if [[ $(echo "if (${VERSION} < 1.5) 1 else 0" | bc) -eq 1 ]] ; then
    # version is lower than 1.5
    ISOK=0 ;
elif [[ $(echo "if (${VERSION} == 1.5) 1 else 0" | bc) -eq 1 && $(echo "if (${SUBVERSION} < 26) 1 else 0" | bc) -eq 1 ]] ; then
    # version is 1.5.x but lower than 1.5.26
    ISOK=0 ;
### and so on - further version checks
else
    ISOK=1 ;
fi

The complete script

Now we are ready to assemble all the parts into a ready-to-use script and add some minor improvements. The script will search for all joomla versions in the given base path and prints information about the status. Of course you cannot speak of a version 1.5.26 as "current" but as it is the most current version of the 1.5 branch and an update to the 2.5 or 3.x branch is very complicated in some cases, this script will mark this version as "OK". You can change this if you like.
Here is a sample output of the script:
[INFO] version 1.5.26 in /var/www/xxx is ok.
[WARN] outdated Joomla version 1.0.12 in /var/www/yyy
[WARN] outdated Joomla version 1.5.14 in /var/www/zzz
[WARN] outdated Joomla version 2.5.8 in /var/www/aaa
[WARN] outdated Joomla version 1.5.10 in /var/www/bbb
And now: the complete script. Just save it as "joomlascan.sh" and call it via
bash joomlascan.sh
If you give a file name as an additional argument (like bash joomlascan.sh list.csv), you get a file called list.csv that contains a list of all outdated installations:
/var/www/yyy;1.0.12;1.5.26
/var/www/zzz;1.5.14;1.5.26
/var/www/aaa;2.5.8;2.5.17
/var/www/bbb;1.5.10;1.5.26
Tip:
If you use ISPConfig 3, you should alter the BASEPATH to BASEPATH="/var/www/clients/client*/web*".
#!/bin/bash

# current version 1.5.x
CUR15=26
# aktuelle version 2.5.x
CUR25=17
# aktuelle version 3.2.x
CUR3=1

#base path of the websites
BASEPATH="/var/www/"

# write to csv file (optional argument)
OUTFILE=$1

if [[ "$OUTFILE" != "" ]] ; then
    # empty CSV file
    echo -n "" > $OUTFILE ;
fi

for L in `find ${BASEPATH} -type d -name 'components' ! -wholename '**/administrator/components' | grep -v '/tmp/'` ; do
    D=`dirname $L` ;
    F=$D/libraries/joomla/version.php ;
    F2=$D/libraries/cms/version/version.php ;
    F3=$D/includes/version.php ;
    ISOK=0 ;
    SHOWNEWEST="" ;
    IMPORTANCE=0 ;
    if [[ -e "$F" || -e "$F2" || -e "$F3" ]] ; then
        if [[ -e "$F" ]] ; then
            F=$F ;
        elif [[ -e "$F2" ]] ; then
            F=$F2 ;
        elif [[ -e "$F3" ]] ; then
            F=$F3 ;
        fi
        VERSION=`grep '$RELEASE' $F | sed -r "s/^.*=\s*'(.*)'.*$/\1/g"` ;
        SUBVERSION=`grep '$DEV_LEVEL' $F | sed -r "s/^.*=\s*'(.*)'.*$/\1/g"` ;
        if [[ $(echo "if (${VERSION} < 1.5) 1 else 0" | bc) -eq 1 ]] ; then
            # version is lower than 1.5
            SHOWNEWEST="1.5.${CUR15}" ;
            IMPORTANCE=3 ;
        elif [[ $(echo "if (${VERSION} == 1.5) 1 else 0" | bc) -eq 1 && $(echo "if (${SUBVERSION} < ${CUR15}) 1 else 0" | bc) -eq 1 ]] ; then
            # version is 1.5.x but not most current version
            SHOWNEWEST="1.5.${CUR15}" ;
            IMPORTANCE=2 ;
        elif [[ $(echo "if (${VERSION} == 1.6) 1 else 0" | bc) -eq 1 ]] ; then
            # version is 1.6
            SHOWNEWEST="2.5.${CUR25}" ;
            IMPORTANCE=2 ;
        elif [[ $(echo "if (${VERSION} == 1.7) 1 else 0" | bc) -eq 1 ]] ; then
            # version is 1.7
            SHOWNEWEST="2.5.${CUR25}" ;
            IMPORTANCE=2 ;
        elif [[ $(echo "if (${VERSION} > 1.7) 1 else 0" | bc) -eq 1 && $(echo "if (${VERSION} < 2.5) 1 else 0" | bc) -eq 1 ]] ; then
            # version is somewhere between 1.7 and 2.5
            SHOWNEWEST="2.5.${CUR25}" ;
            IMPORTANCE=2 ;
        elif [[ $(echo "if (${VERSION} == 2.5) 1 else 0" | bc) -eq 1 && $(echo "if (${SUBVERSION} < ${CUR25}) 1 else 0" | bc) -eq 1 ]] ; then
            # version is 2.5 but lower than current
            SHOWNEWEST="2.5.${CUR25}" ;
            IMPORTANCE=1 ;
        elif [[ $(echo "if (${VERSION} >= 3) 1 else 0" | bc) -eq 1 && $(echo "if (${VERSION} < 3.2) 1 else 0" | bc) -eq 1 ]] ; then
            # version is 3.0 or 3.1
            SHOWNEWEST="3.2.${CUR3}" ;
            IMPORTANCE=2 ;
        elif [[ $(echo "if (${VERSION} == 3.2) 1 else 0" | bc) -eq 1 && $(echo "if (${SUBVERSION} < ${CUR3}) 1 else 0" | bc) -eq 1 ]] ; then
            # version is 3.2 but lower than current
            SHOWNEWEST="3.2.${CUR3}" ;
            IMPORTANCE=1 ;
        else
            ISOK=1 ;
            echo "[INFO] version $VERSION.$SUBVERSION in $D is ok." ;
        fi
    else
        # seems not to bee a joomla version ...
        ISOK=1 ;
    fi ;
    
    if [[ $ISOK -eq 0 ]] ; then
        echo "[WARN] outdated Joomla version $VERSION.$SUBVERSION in $D" ;
        if [[ "$OUTFILE" != "" ]] ; then
            # write CSV file
            echo "\"$D\";$VERSION.$SUBVERSION;$SHOWNEWEST;$IMPORTANCE" >> $OUTFILE ;
        fi
    fi
done

exit 0 ;
There is one known issue whith this script: it cannot recognize Mambo. So it will mark all mambo installations as "OK" regardless which version they have.

Wednesday, April 01, 2015

ruby - Testing STDOUT output in Rspec - Stack Overflow

ruby - Testing STDOUT output in Rspec - Stack Overflow



I am trying to build a spec for this statement. It is easy with 'puts'
print "'#{@file}' doesn't exist: Create Empty File (y/n)?"
expect { my_method }.to output("my message").to_stdout
expect { my_method }.to output("my error").to_stderr
out, err = capture_io do
  my_method
end

assert_equals "my message", out
assert_equals "my error", err
The code below is no longer required, unless you work with Rspec < 3.0 or a different framework.

This code here will allow you to capture whatever is sent to stdout and stderr, respectively:
require 'stringio'

def capture_stdout(&blk)
  old = $stdout
  $stdout = fake = StringIO.new
  blk.call
  fake.string
ensure
  $stdout = old
end

def capture_stderr(&blk)
  old = $stderr
  $stderr = fake = StringIO.new
  blk.call
  fake.string
ensure
  $stderr = old
end
Now when you have a method that should print something to the console
def my_method
  # ...
  print "my message"
end
you can now write a spec like this:
it 'should print "my message"' do
  printed = capture_stdout do
    my_method # do your actual method call
  end

  printed.should eq("my message")
end

Monday, March 30, 2015

jquery - Angular JS: load custom javascript file in ng-view - Stack Overflow

jquery - Angular JS: load custom javascript file in ng-view - Stack Overflow



With helps of ngRoute a template can be loaded which is containing a script tag which includes the js file contains javascript codes that we wanted to run.
Please take a look the below example for usage:

Thursday, March 12, 2015

Full width slider in prestashop 1.6

Full width slider in prestashop 1.6



In this tutorial i want to show you how easily you can change the way of how homeslider appears in default-bootstrap template inPrestaShop 1.6. As you know default homeslider module is a part of homepage of your shop. It appears right below the main menu. Near the module (on right hand side) you can see two small banners. In this guide i will show you how to remove these banners, and how to change width of slider to 100%.

Full width slider default bootstrap prestashop 1.6


How to change full width to 100%
In guide related to prestashop 1.5: homeslider to top ful width we had to change a lot of things in prestashop files, like stylesheet, like template file and also module core (to support new hook). In prestashop 1.6 we can achieve all of this without touching files. We can do everything in back office with available settings and features. Awesome!

Disable to small banners on the right hand side
First step in our modificaiton: we have to remove two small banners on the right hand side of the main slider. These images are a part of "theme configurator" module and to remove these banners we just have to disable this module. But be aware, don't disable module at all, just disable it in hook named displayTopColumn. In this case go to modules > positions tab in back office. Search there for modules list named displayTopColumn. You will see there two modules (by default there are two modules). Click on edit button near theme configurator and click on "unhook", exactly as i show on image below.

Unhooking theme configurator module from displayTopColumn position
unhook theme configurator for prestashop

After this action two small baners will disappear but they will left blank empty space. So we have to fill this blank space with our slider module. Then it will have full width :) 

Change width of the banner
Go to modules > modules tab and search for "image slider for your homepage" addon. Click on configure button near this module and then - you will be redirected to module configuration page. In the main "settings" form change Max width param value from 799px to 1170px (just type 1170 in this field exactly as i show below). Wondering why 1170? 1170 is a width of the main page container.

changing width of the default home slider module from 799 to 1170
homeslider configuration page prestashop 1.6


Removing banners with width smaller than 1170
Due to the fact that we changed width of the slider window, we have to remove old smaller than 1170px (width) pictures that we uploaded. We have to upload pictures with min-width: 1170px; On screenshot below i show how to remove actual slides. Below the "settings" form (where you changed width of slider) you can manage already uploaded slides. Just hit on delete button like i show:

removing default slides with old small width value (799)
remove slides prestashop 1.6

Upload images with new sizes
If you removed all old slides with wrong width - now it's time to upload new pictures. So, you have to prepare own slides for example in photoshop or other desktop software. Create slides with width at least 1170 and upload them to slider. They will appear as a "full" width slides, like i show on effect image at the bottom of this article. After upload new slides - don't forget to Enable them :) You can download example of image with 1170 width here: download picture

New slide uploaded to website
new slides with correct width


effect of our modifications: full width slider in prestashop 1.6
full width slider in prestashop 1.6

Friday, February 20, 2015

Boban Acimovic Consulting | Joining non-related tables in Propel 1.6

Boban Acimovic Consulting | Joining non-related tables in Propel 1.6



I had this problem few days ago and I couldn’t find any useful information on the Internet. I had to browse through Propel internals to find this out and it would be pity not to share it with others. So let’s start with an example:

$accounts = AccountQuery::create()
->joinWith('Account.AdditionalData')
->addJoin(AdditionalDataPeer::VAL, CountryPeer::ID)
->withColumn('country.name', 'CountryName')
->find();
Someone may ask why Country and Account objects have no relation. Well, this is just an example, but imagine that table AdditionalData contain many different values, so we can’t really make any foreign reference as it would have to reference many different tables, which is not possible. There may be other reasons to use something like this, but anyway let’s explain line by line how it works.

->joinWith('Account.AdditionalData')
This is just normal Propel join where we hydrate the main object with related object data.

->addJoin(AdditionalDataPeer::VAL, CountryPeer::ID)
This is the way how we can define additional join with non-related table. The first value, AdditionalDataPeer::VAL represents the field name in the left table and CountryPeer::Id it’s corresponding field in the right table. These constants are defined in base peer classes. As third parameter here you can define the type of join, but I have used just default (inner join). This works fine except the main object is not hydrated with the related object data. Unfortunately, this is not possible using with() method as with() works only with previous join(). Fortunately, there is another way using:

->withColumn('country.name', 'CountryName')
First parameter here is the real table name concatenated with real column name and the second one is an alias name for this column. You can include as many columns from the related table as you want. It’s probably possible to concatenate some predefined Propel constants for the first parameter, like CountryPeer::TABLE_NAME . ‘.’ . CountryPeer::ID, but I haven’t tried that.
And like the documentation for withColumn() says, you can use this value in the resulting object by method getVirtualColumn():

foreach ($accounts as $account) {
print $account->getVirtualColumn('CountryName') . PHP_EOL;
}
I hope this may help someone :)