Solution Resort

Learn, try and share…

Author: Richard (Page 2 of 3)

Mount Shared Folder to Ubuntu if auto-mount doesn’t work

I had some issues with mounting shared folders with Virtual Box because they don’t appear anywhere. I finally figured out that the Shared drive is already there, it’s just not showing anywhere.

Before continuing, making sure you have installed the latest version of Guest Additions and your user is added to the vboxsf group.

First, add a shared folder under Devices -> Shared Folder Settings with the following properties (I am using windows in this example),

Folder Path: C:your-folder
Folder Name: shared-name-on-ubuntu

Go to a folder, e.g. ~/ and run

mkdir ~/shared-name-on-ubuntu

Finally mount it

sudo mount -t vboxsf shared-name-on-ubuntu ~/shared-name-on-ubuntu

And now you should be able to see everything shared within Ubuntu.

 

Create a Zend Framework 2 Module for Composer

I am sure everyone has been using Zend Framework 2 (ZF2) modules created by many others via Packagist, but it can be a little tricky when it comes down to creating your own ZF2 modules to work with Composer, especially when using your own private VCS repository. I found the documentation for setting this up is rather difficult to find so I thought it might be useful to pull everything together.

Let’s get started.

First, we need to create a ZF2 module.  To save a bit of time, I have already created a skeleton ZF2 module on Github which you can just grab. This is a typical ZF2 module with an additional composer.json file.  composer.json is required by Composer to specify the configuration options. The minimum you’ll need is “name”, “require” and “autoload”. The “name” section is used to tell Composer what your module is called and it is a unique identifier for your module . Please use a unique namespace to avoid conflicting with other projects. The “require” section is used to specify dependencies. For example, if your module is depending on ZF2, you’ll need to include it there. Finally the “autoload” section is used to define the PHP namespace for your module. This is explained very well by Composer. Without specifying the autoload section, Composer won’t generate an autoloader for the root namespace of your module and therefore ZF2 won’t know where to find it. In addition to the “psr-0” section, I have also added a “classmap” section including the path to “Module.php”. This is required because Module.php within a ZF2 module is outside the src folder, in order to make ZF2 aware of it, we’ll need to add it separately.

We have created our first ZF2 module that works with Composer. Now we need to see this in action like how we use all other open source Composer packages. The key difference here is we are likely to have our ZF2 module stored within a private VCS. To use it with Composer, it requires a special section added to composer.json file within our ZF2 Application.

"repositories": [
 {
 "type": "vcs",
 "url" : "git@github.com:shenghuahe/zf2-composer-module.git"
 }
 ]

I am using a Git repository on Github here. This section is required to tell Composer where to find your ZF2 module. And then, you’ll need to include it under the “require” section,

"require": {
 "shenghuahe/zf2-composer-module" : "dev-master"
 }

“dev-master” is pointing to your master branch. You can always make a tag and point it to the tag instead.

That’s it. Run php composer.phar update and see it in action!

 

Other things to keep in mind

  • If you are using a private repository, you’ll need to authenticate with the server. I haven’t done this with SVN, but with GIT, all you have to do is to generate & install a SSH public/private key pair on your OS and copy & paste the public key to your GIT server. The instruction can vary depending on the host you choose. Here is the instruction for Bitbucket https://confluence.atlassian.com/display/BITBUCKET/Set+up+SSH+for+Git.

Configure AngularJS server

In the Start an Angular project with Yeoman (Ubuntu 13.4) article (please read this one first or you may have problem following this article), I explained how to start an angular project with Yeoman as well as using Grunt to manage the work flow of your application. However you may wonder when making http requests to the server (back-end), where is that server and how it should be configured. I’ll cover all of that in this article. 

You can use anything for your back-end code. NodeJS with express, Zend Framework, Rails or anything else you prefer. The concept is the same. However if you are using Apache as the web server, you will have to add a virtual host for your back-end code. In this article, I’ll use NodeJs with express as the example. 

Luckily there is Yeoman generator for NodeJs with Express. To install it, simply run the following command in your terminal (I usually install generators on the global scope for convenience )

sudo npm install -g generator-node-express

My personal preference is to leave the angular client side code and the service side code in the same repository. However, they do need to be separated because they are two separate applications. I typically have the following folder structure,

YourApp/client #angularJS client side code
YourApp/server #nodeJS with express service side code

To generate the AngularJs client side scaffolding, you’ll need to run yo angular within the client folder. To create the server side code using Yeoman, simply run yo node-express within the server folder.

You’ll be asked a few questions when installing node-express, but because we don’t need any angular stuff nor the SASS parser, I recommend unticking those options when installing. If you had issues installing, please refer to the official documentation on NPM (https://npmjs.org/package/generator-node-express)

Now we need to start the node server. If you don’t want the faff, just go to YourApp/server and type in

node app.js

This will start your http server on localhost:3000. If you wish to automate the workflow (i.e. automatically restart the server when any watched files are modified, use grunt restart instead).

We now got to the point that we have a server and we can open the browser and type in http://127.0.0.1:3000 to see our server side home page. How exciting! However remember our AngularJS app has been setup on port 9000. Making calls from 127.0.0.1:9000 to 127.0.0.1:3000 is going to cause problems because cross domain traffic is blocked by default unless you deliberately allow it.

To solve this problem, we need to setup a proxy. In other words, this means when accessing 127.0.0.1:9000/api, we’ll tell the server go via 127.0.0.1:3000/api instead behind the scene.

This is done by using the grunt-connect-proxy module which has a very good instruction on https://npmjs.org/package/grunt-connect-proxy. Although this is a indeed very good instruction, I do have a few things to point out because I got stuck on this for a few hours due to the lack understanding of grunt.

  • Under the connect:proxies section, the context has a value of ‘cortex’. This  means your 127.0.0.1:9000/cortex route is pointing to 127.0.0.1:3000/cortex not 127.0.0.1:3000. I thought it was going into the top level route but apparently it didn’t.
  • The next thing is it’s really useful to use the –verbose option when running grunt server. i.e. grunt server –verbose. This is going to give you very detailed output when starting the server and on top of that, you’ll also see access logs in the terminal. i.e.
Proxied request: /api/home -> http://localhost:3000/api/home
{
    "host": "127.0.0.1:9000",
    "connection": "keep-alive",
    "content-length": "2",
    "accept": "application/json, text/plain, */*",
    "origin": "http://127.0.0.1:9000",
    "user-agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.57 Safari/537.36",
    "content-type": "application/json;charset=UTF-8",
    "referer": "http://127.0.0.1:9000/",
    "accept-encoding": "gzip,deflate,sdch",
    "accept-language": "en-GB,en-US;q=0.8,en;q=0.6",
    "cookie": "_ga=GA1.4.194784887.1384818518"
}

That’a pretty much it. Grab it and have fun!

 

 

Start an Angular project with Yeoman (Ubuntu 13.4)

This guide uses Yeoman to scaffold  an Angular Application. It should get you started in no time.

————————–

Please note that DO NOT use sudo to run any of the npm commands. npm will try to use the ~/tmp folder to create temporary files, this folder should belong to the user currently logged in, it can cause all sorts of permissions issues if you use the sudo command to run any npm tasks. However if you have done this in the past, the best thing to do is to recusively set everything within your home directory to the correct permission. Such as

sudo chown -R <username> ~/*
sudo chgrp -R <username> ~/*

————————–

Install Yeoman

npm install -g yo

Install Angular generator plug-in

npm install -g generator-angular

Install Rudy and Compass [optional: if you want to use SASS].

sudo apt-get install ruby -y
sudo apt-get install compass -y

*There is also a good article talking about why SASS is good.

Create the project

mkdir <project-folder-name> && cd $_
yo angular

*This will only generate the bare minimum skeleton for your application, if you need to generate controllers, views, templates, read the guide on github.

To build the project for production

grunt

To preview the project

grunt server

*This is the genius part. When you start the server, all changes you made to the application will be instantly reflected into the browser. This is done by configuring the ‘watch’ section within gruntfile.js

At last, I strongly suggest you watch the introduction videos on the Yeoman website.

Ant returns code 137

I had this problem at work when running php unit tests on Jenkins (CI server) and it took the whole team ages to figure out this problem. I thought it’s something worth sharing.

We use Jenkins to run ant which invokes our unit tests and to speed up the process we are running all the unit tests in one single process (this means we are not doing phpunit –process-isolation which runs each test in a separate PHP process. The up side of using this option is it uses a lot less memory, but the down side is it’s significantly slower). The whole process consumes around 500 mb of memory in one single process but one day we started getting the following error,

BUILD FAILED
<http://jenkins/project/build.xml>:245: The following error occurred while executing this line:
/path..to..build/build.xml:66: exec returned: 137

We thought it might be something related to memory but we weren’t sure about it until one day we checked the kernel log (by typing in dmesg) and we get,

Out of memory: Kill process 16394 (phpunit) score 193 or sacrifice child
Killed process 16394, UID 497, (phpunit) total-vm:791308kB, anon-rss:348416kB, file-rss:4kB

And we know it’s memory related for sure, after increased the virtual memory allowance to 4G, it solved the problem.

However if you don’t have the luxury to increase your memory limit, try to use the –process-isolation option which would only use a tiny amount of memory to run all the tests but I found this takes 5-6 times longer to run all tests.

 

Hope this helps.

Debian Zero To Hero Guide (best for PHP developers)

Took me quite a while to pull all these information together. This guide should install most basic packages you will need to get started with PHP Development.

*This was tested with Debian 7.1.0

Before we start, please launch a root terminal to save typing a million sudo commands. Short cut key: Alt + F2, search for terminal and run Sudo Terminal

Java First (Get rid of Open JDK and install Oracle JDK)

Found these from (http://www.webupd8.org/2012/06/how-to-install-oracle-java-7-in-debian.html)

echo “deb http://ppa.launchpad.net/webupd8team/java/ubuntu precise main” | tee -a /etc/apt/sources.list
echo “deb-src http://ppa.launchpad.net/webupd8team/java/ubuntu precise main” | tee -a /etc/apt/sources.list
apt-key adv –keyserver keyserver.ubuntu.com –recv-keys EEA14886
apt-get update
apt-get install oracle-java7-installer

#install this package to automatically set the Java Environment variables
apt-get install oracle-java7-set-default

Basic Stuff

This includes Apache2, PHP5 ,Curl for PHP, Ant, PHP Internationalisation, Tidy, MySQL, PHP-MySQL, Gnome system tools

apt-get install apache2 -y
apt-get install php5 -y
apt-get install php5-curl -y
apt-get install ant -y
apt-get install php5-intl -y
apt-get install php5-tidy -y
apt-get install mysql-server mysql-client -y
apt-get install php5-mysql -y
apt-get install ssh -y
apt-get install gnome-system-tools -y

Apache Authentication

a2enmod auth_digest //enable digest authentication module

Image libraries

apt-get install php5-imagick
apt-get install php5-gd

MySQL Optimization for InnoDB

This will significant increase the query time for InnoDB but it’s important to leave this as 1 on a production environment if you wish MySQL to write to the log on every commit. For more information, see http://dev.mysql.com/doc/refman/4.1/en/innodb-parameters.html

innodb_flush_log_at_trx_commit = 0

Add a user to the sudo group

The registered user doesn’t come under the sudo group by default, if you wish to install packages or run sudo commands with the current user, add the user to the sudo group by typing in,

adduser <username> sudo

You must re-login for this to take effect

Unmanaged Network Issue

You may encounter this issue when setting up multiple network adapters (the direct impact would be internet connection doesn’t work). This is usually down to this file /etc/network/interfaces having instructions telling the network manager that you want to manage eth0 manually.

To fix it, simply go to this file and make sure the following lines are commented out

#allow-hotplug eth0
#iface eth0 inet dhcp

Install Samba

apt-get install libcupsys2 samba samba-common

To share a directory, edit /etc/samba/smb.conf and add the following lines to the bottom of the file,

[home]
comment = debian.www
path = /home/<username>/www
browseable = yes
valid users = @<username>
writable = yes
create mask = 0700
directory mask = 0700

Restart Samba by typing in

service samba restart

More to follow…

Prevent Cross Site Scripting (XSS) attacks from Rich Text Editors

I believe most developers already know the importance of converting / escaping special HTML characters when displaying their content on a Website to prevent XSS attacks.  In PHP, this can be simply done by calling htmlspecialchars([dangerous data]).  However I haven’t seen many articles talking about how to do this when you need to use a Rich Text Editor on your Website and needed the user to enter html (like a blog).

The easiest way to achieve this is by removing unwanted html tags such as <script>, <iframe> or <embed>. There is a really good PHP library called HTMLPurifier which is just designed to solve this problem.

To use it, you can either download the latest archive from HTML Purifier’s official website or if you are using a framework (such as ZF2 or Symphony 2), you can get it via Composer (https://packagist.org/packages/ezyang/htmlpurifier). Once it’s installed, you can use the following code to purify your html data.

$config = HTMLPurifier_Config::createDefault();

//the following line is optional, this is to turn off the caching. If you prefer better performance by leaving the caching on, the permission of the htmlpurifier/library/HTMLPurifier/DefinitionCache/Serializer folder will need to be made 777.
$config->set('Cache.DefinitionImpl', null);
$purifier = new HTMLPurifier($config);

//just replace $rawHTML with your Raw HTML data from the user input
$clean_html = $purifier->purify($rawHTML);

Html Purifier mainly does two things.
First, it tries to fix the HTML and guarantee the output is Standards Compliant, E.g.

<p>test

will be converted to

<p>test</p>

The next thing it does is to remove malicious html tags to prevent XSS attacks, E.g.

<script>alert(0);</script>
<p>line1</p>
<iframe src="somedangerousURL.com"></iframe>

will be converted to

<p>line1</p>

This is a great tool and it will make you site a lot more secure by keep it away from common XSS attacks. For more information and configuration options, check the official HTML Purifier Website

p.s.  Some may find the output contains weird characters such as “”, after the html gets purified. This is not a problem with HTML Purifier but is usually the problem with your encoding. There are usually a few things you need to check,

  • Encoding of PHP. E.g. 
    header('Content-Type:text/html; charset=UTF-8');
  • If you store the purified html into a database. Check encoding of your Database Access Layer (i.e. if you are using PDO directly) or your Database Abstraction Layer (i.e. Doctrine). Please note that the Database itself will also need to be set to UTF-8. I use Doctrine 2, and if you don’t know where to set the encoding, it’s next to all your database credentials.
		'user' => [username],
                'password' = [password],
                'dbname' => [dbName],
                'host' => [host],
                'charset' => 'utf8'
  • To ensure the website displays all none ASCII characters correctly, add the following meta tag
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />

MySQL Integer Types

When declaring an integer in MySQL the number after the integer does not have any bearing on the size of the information in the database or the range of possible values.

 

INT(1) uses the same space as INT(11)

 

If you want to specify a smaller / different data size, this can be done with:

TINYINT  = 1byte (256 possible values)

SMALLINT = 2bytes  (65535 possible values)

MEDIUMINT = 3 bytes (16777215)

INT = 4 bytes (4294967295)

BIGINT = 8bytes (18446744073709551615 possible values!!)

 

The value after the type declaration INT(11), is purely an instruction suggesting to MySQL how many zero digits to left pad the number with for display.  e.g. 00000000001.   In PHP/MySQL it has no effect and is best left to the defaults.

 

To read more about this, go to

http://dev.mysql.com/doc/refman/5.0/en/numeric-types.html

 

Ubuntu Zero To Hero Guide (best for PHP developers)

Took me quite a while to pull all these information together. This guide should install most basic packages you will need to get started with PHP Development.

*This was tested with Ubuntu 13.04 but most things should also work on Ubuntu 12.

Java First

#remove open jdk and install oracle jdk

sudo apt-get purge openjdk*

sudo apt-get autoremove

sudo add-apt-repository ppa:webupd8team/java

sudo apt-get update

sudo apt-get install oracle-java7-installer

 

Basic Stuff

sudo apt-get install apache2 -y

sudo apt-get install php5-curl

sudo apt-get install ant -y

sudo apt-get install php5-intl

sudo apt-get install php5-tidy

sudo apt-get install mysql[*] (type in mysql and it will tell you what version to install)

sudo apt-get install mysql-server -y

sudo apt-get install php5 -y

sudo apt-get install php5-mysql -y

sudo apt-get install ssh -y

sudo apt-get install gnome-system-tools -y

#the PECL_HTTP extension
sudo apt-get install libcurl3-openssl-dev -y
sudo pecl install pecl_http -y
#when this is done, add “extension=http.so” in /etc/php/apache2/php.ini under the “Dynamic Extensions” section

Apache Authentication

sudo a2enmod auth_digest //install digest authentication module

Image libraries

sudo apt-get install php5-imagick
sudo apt-get install php5-gd

#I think it makes sense not to use imagick and gd directly but to use Imagine which is a wrapper for the two available on packagist. It has a much user friendly API which I highly recommend if you want a time saver.
Here is the link to it on packagist https://packagist.org/packages/imagine/imagine

 

MySQL Optimization for InnoDB, (This will significant increase the query time for InnoDB but it’s important to leave this as 1 on a live environment. For more information, see http://dev.mysql.com/doc/refman/4.1/en/innodb-parameters.html)

innodb_flush_log_at_trx_commit = 0

#To debug slow queries, set the following configuration options

log_queries_not_using_indexes = on
slow_query_log = on
log_output = table

and then look at the ‘slow_query’ table in the ‘mysql’ database.

Install MongoDB

sudo apt-key adv –keyserver hkp://keyserver.ubuntu.com:80 –recv 7F0CEB10
echo ‘deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen’ | sudo tee /etc/apt/sources.list.d/mongodb.list
sudo apt-get update
sudo apt-get install mongodb-10gen

#gui tool
http://robomongo.org/

Samba

If your Ubuntu is installed as a Virtual Machine, it’s very likely you’ll come cross using Samba to share folders so that a mapped drive can be created in your native OS to access them directly.

The easiest way to do this is by right clicking on a folder and select Sharing Options, then tick the Share this folder box and the Allow others to create and delete files in this folder box. Save your changes.

Once this is done, you’ll need to manually create a Samba user to gain access to the shared folder. This can be simply done by,

sudo smbpasswd -a <username>

 

Xdebug

#xdebug

sudo apt-get install php5-dev -y

sudo apt-get install php-pear -y

sudo pecl install xdebug

#then add the following lines into /etc/php5/apache2/php.ini

[XDebug]

zend_extension=”/usr/lib/php5/[*date*]/xdebug.so”

xdebug.remote_enable=1

xdebug.remote_handler=dbgp

xdebug.remote_mode=req

xdebug.remote_host=127.0.0.1

xdebug.remote_port=9000

#finally make sure xdebug shows in phpinfo();

Xdebug – CMD setup

edit /etc/php5/apache2/php.ini and add the following section on the bottom,

[XDebug]

zend_extension=”/usr/lib/php5/[*date*]/xdebug.so”

xdebug.remote_enable=1

xdebug.remote_handler=dbgp

xdebug.remote_mode=req

xdebug.remote_host=127.0.0.1

xdebug.remote_port=9000

I am using Netbeans as my IDE, Type in

export XDEBUG_CONFIG=idekey=netbeans-xdebug on command line and start Netbean debug session to listen on the same key.

Now, simply run phpunit <Test Script Name>.php should start the debug session.

If you need to stop debugging from command line, type in
unset XDEBUG_CONFIG

 

Rabbitvcs (svn client gui on linux)

http://wiki.rabbitvcs.org/wiki/install/ubuntu

#the following line is a fix for utuntu 13.04 (please note that in Ubuntu 13.04, you can only bring up the GUI from command line. E.g. rabbitvcs commit)

sudo ln -s /usr/lib/x86_64-linux-gnu/libpython2.7.so /usr/lib/libpython2.7.so.1.0

 

Firewall GUI tool

search gufw from software manager

 

VHost Setup

First, follow the guide from

https://www.digitalocean.com/community/articles/how-to-set-up-apache-virtual-hosts-on-ubuntu-12-04-lts

Just a quite note, to enable or disable a vhost,

Enable: sudo a2ensite [site-name]

Disable: sudo a2dissite [site-name]

And,

#Enable rewriteengine

a2enmod rewrite

In order to use the .htaccess file, you need to set

AllowOverRide All within the Apache config file

Nodejs

#install node

sudo apt-get install g++ curl libssl_dev apache2-utils git-core
sudo apt-get install python-software-properties
sudo add-apt-repository ppa:chris-lea/node.js
sudo apt-get update
sudo apt-get install nodejs

If these commands don’t work, check http://slopjong.de/2012/10/31/how-to-install-the-latest-nodejs-in-ubuntu/

Bower (A Package Manager for the web)

sudo apt-get install npm

Then follow https://github.com/bower/bower

 

Install PHPUnit

Firstly, run:

sudo pear channel-discover pear.symfony.com
sudo pear install pear.symfony.com/Yaml

Then run:

sudo pear channel-discover pear.phpunit.de

//this may take a while depending on your internet speed
sudo 
pear install –alldeps pear.phpunit.de/PHPUnit

p.s.

http://pear.phpunit.de/ is the most up to date official PHPUnit site. If you install it from the default pear package, it will be version 3.4 or 3.5 which has a lot of the useful features missing.  It’s possible this guide will fail because there are some dependencies missing. If that happens, follow the instructions on your screen to install the missing packages first (e.g. php-dom).

Page 2 of 3

Powered by WordPress & Theme by Anders Norén