serve downloads through php with pause/resume support

Posted May 11th, 2010 in Uncategorized by pixelbyter

Using PHP to serve protected files to specific web users can be very useful for the developer, but the end user usually misses out as functionality such as download resuming is often missed out.

I’ve put together the following function which will serve files to the user with support for download resuming after they’ve been paused or the connection has been dropped. Also, by serving the data in 1MB chunks this function is very low on memory usage.

function dl($file_location, $file_name) {
	try {
		//check if file exists
		if(!file_exists($file_location))
			throw new Exception('Could not find file');

		//file size
		$file_size = filesize($file_location);

		//open download file
		$open_download_file = fopen($file_location, 'rb');
		if(!$open_download_file)
			throw new Exception('Could not open file');

		//download range
		$start_point = 0;
		$end_point = $file_size - 1;

		//partial download
		if(isset($_SERVER['HTTP_RANGE']) && !empty($_SERVER['HTTP_RANGE'])) {
			//download range
			$http_range = explode('-', substr($_SERVER['HTTP_RANGE'], strlen('bytes=')));
			$start_point = $http_range[0];
			if($http_range[1] > 0)
				$end_point = $http_range[1];

			//headers
			header('HTTP/1.0 206 Partial Content');
			header('Status: 206 Partial Content');
			header('Accept-Ranges: bytes');
			header('Content-Range: bytes '.$_SERVER['HTTP_RANGE'].'/'.$file_size);
			header('Content-Length: '.($end_point - $start_point + 1));
		}
		//full download
		else {
			//headers
			header('Content-Length: '.$file_size);
		}

		//headers
		header('Content-Type: application/force-download');
		header('Content-Disposition: attachment; filename="'.$file_name.'"');
		header('Last-Modified: '.date('D, d M Y H:i:s \G\M\T', filemtime($file_location)));

		//headers for ie6 & ie7
		header('Expires: 0');
		header('Pragma: public');
		header('Cache-Control: must-revalidate');

		//jump ahead in file to start
		if($start_point > 0)
			fseek($open_download_file, $start_point);

		//serve data chunk and update download progress log
		$download_bytes_position = $start_point;
		while(!feof($open_download_file) && $download_bytes_position <= $end_point) {
			//mark download as aborted if connection has been closed
			if(connection_aborted() || connection_status() != 0)
				throw new Exception('Connection aborted');

			//calculate next chunk size
			$download_data_chunk_size = 1048000;
			if($download_bytes_position + $download_data_chunk_size > $end_point + 1)
				$download_data_chunk_size = $end_point - $download_bytes_position + 1;

			//get data chunk
			$download_data_chunk = fread($open_download_file, $download_data_chunk_size);
			if(!$download_data_chunk)
				throw new Exception('Could not read file');

			//output it
			print($download_data_chunk);
			flush();

			//increment download point
			$download_bytes_position += $download_data_chunk_size;
		}

		//close archive file
		fclose($open_download_file);
	}
	catch(Exception $e) {
		//any error handling you need to do should go here
		return false;
	}

	return true;
}

automate rootkit scans for your web server

Posted April 30th, 2010 in Uncategorized by pixelbyter

If your system becomes infected by a rootkit then someone can corrupt or steal your data, deface your sites, steal your data, send spam or launch phishing attacks. This article will help you set up automated scans which email you about any issues on your server. I have written guides for chkrootkit and rkhunter, two great rootkit scanners, both of which I would recommend running as each excel in different areas.

chkrootkit

Install and run chkrootkit by running the following commands (as root or su).

wget ftp://ftp.pangeia.com.br/pub/seg/pac/chkrootkit.tar.gz
tar xvzf chkrootkit.tar.gz
cd chkrootkit*
make sense
./chkrootkit

Create a new daily cron job to run the scan (this code uses the vi text editor).

cd /etc/cron.daily
vi chrootkit.sh

Enter the following text (this code assumes chkrootkit was installed to /root/chkrootkit-0.49).

cd /root/chkrootkit-0.49/
./chkrootkit | mail -s "chkrootkit scan results" YOUR_EMAIL_ADDRESS

rkhunter

Install and run rkhunter by running the following commands (as root or su). At the time of writing 1.3.6 was the latest version of rkhunter.

wget http://downloads.sourceforge.net/project/rkhunter/rkhunter/1.3.6/rkhunter-1.3.6.tar.gz?use_mirror=nchc
tar xvzf rkhunter-1.3.6.tar.gz
cd rkhunter-1.3.6
./installer.sh ––install
./rkhunter --update
./rkhunter -c

Create a new daily cron job to run the scan.

cd /etc/cron.daily
vi rkhunter.sh

Enter the following text (this code assumes chkrootkit was installed to /root/rkhunter-1.3.6).

cd /root/rkhunter-1.3.6/files/
./rkhunter --cronjob --rwo --nocolors | mail -s "rkhunter scan results" YOUR_EMAIL_ADDRESS

Scratch My Back iTunes LP

Posted February 16th, 2010 in Uncategorized by pixelbyter

Scratch My Back and the Scratch My Back iTunes LP (get a sneak peak here) have been released. The iTunes LP is the first I’ve ever built and it’s already the #1 selling in Germany. This is just one of many to be released from our base in Box in the coming months.

optimising MySQL server

Posted February 9th, 2010 in Uncategorized by pixelbyter

Over the weekend the Real World web sites went down following a torrent of visitors with the imminent release of Peter Gabriel’s Scratch My Back, which played havoc on our MySQL databases and downed our websites. Our connections maxed out and started to queue, causing any pages requiring a DB connection to refuse to load. After spending the weekend altering MySQL’s settings the average number of simultaneous connections on our MySQL database dropped from 100 to about 2 thanks to the vastly increased speed at which queries are now running. Here’s the top 3 things that were changed to speed things up (keep in mind, our web server currently runs both MySQL and Apache with 8GB RAM):

query_cache_size

The MySQL query cache will cache the result sets of your queries, serving them instead of executing the same queries again. According to MySQL, “Searches for a single row in a single-row table are 238% faster with the query cache than without it.” For a query to return a cached result it must match the cached query exactly, in terms of case and whitespace, and it must not contain subqueries (read this article about Rewriting Subqueries as Joins). This is disabled by default in MySQL, so be sure to enable it first. I set it to 64MB.

query-cache-type = 1
query_cache_size = 64M

key_buffer_size / innodb_buffer_pool_size

The key buffer size is the amount of memory allocated to store table indexes, which can considerably speed up a query as searching for which rows to retrieve may require no disk access. innodb_buffer_pool_size is the innoDB implementation of this setting. On dedicated MySQL servers it is recommended to set this to 75% of the amount of RAM installed. I set key_buffer_size to 256MB as we have few MyISAM tables, and innodb_buffer_pool_size to 1GB so Apache and other services were not adversely affected.

key_buffer_size = 256M
innodb_buffer_pool_size = 1G

table_cache

The table cache is the number of tables that will be stored in memory at once, each table being stored in the cache as MySQL accesses it. I set it to 256.

table_cache = 256

helpful apps

MySQL Report will return information on how your server is currently performing and a list of some of the more useful variables. Get MySQL Report here, and get a guide to understand the report here.

MySQL Tuner will analyse your server and provide recommendations as to how different MySQL settings could be altered and any problems that could result from the current configuration. Get MySQL Tuner here.

developing iTunes LPs/iTunes Extras for the AppleTV

Posted January 25th, 2010 in Uncategorized by pixelbyter

Apple’s ‘hobby’ TV unit, AppleTV, is a rather limited platform in terms of interactability and processing power. After getting my hands on a unit for testing iTunes LPs on, I was shocked at the changes necessary in order to make an LP work on this medium. This article does not cover the information outlined within the iTunes Extras/iTunes LP Development Guide, which features a section covering some of the basics for AppleTV development.

designing for the AppleTV remote

Navigation is limited to the 6 input controls on the AppleTV remote: up, down, left, right, select/play/pause and back/menu. There is no way to override this and use a mouse cursor as the META tag in the projects main web file, index.html, would imply (<meta name=”hdtv-cursor-off” content=”true”/>). TuneKit, Apple’s LP/Extras Javascript development framework, automatically implements methods of controlling your project via the AppleTV remote. When using the default TKController up and down will be automatically used to scroll within a scrollable content box. Although this is helpful in a way, it then leaves the only methods of navigation around the interface of the left and right buttons, which can will leave any vertical navigation inopperable. Horizontal navigation or no navigation whatsoever is best for scrolling content pages. iTunes disables the keyboard interface, but Safari will let you use the cursor keys for directional movement, enter for select/play/pause and backspace for back/menu, so try navigating through your project using just these keys during development.

developing for limited processing power

The AppleTV is quite limited when it comes to what it can comfortably process when it comes to Javascript and CSS transitions. CSS transitions are always preferable to Javascript ones within this environment, thanks to how CSS3 transitions are optimised on the webkit platform, so try to avoid using jQuery for eye candy. Even when using CSS transitions you will notice that they refresh slowly, making fades and movement look jumpy. Try to use longer transition times to minimise this effect, or no transitions at all if possible. Many LPs/Extras disable visualisers and other effects dependant upon Javascript for the AppleTV.

designing for display on a television set

The AppleTV can display resolutions as low as 420p, but don’t let that worry you. All LP/Extra content should be designed for an HD TV at 1280×720 pixels. The title-safe display area is 1024×576, but very few LPs/Extras I have found have limited themselves to this area. Colour and brightness levels can vary greatly between TV sets, much more than on computer monitors. If you are using subtle colour or alpha changes in your navigation try to make them much more prominant on the AppleTV.

beware

The AppleTV uses a custom version of the Safari code, and so operates slightly differently in places. Meaning it has a couple of bugs. Test everything throroughly on an AppleTV before deployment. Just because something works in iTunes and Safari it doesn’t mean that it will work on the AppleTV.

in conclusion

  • Only use horizontal navigation on pages using a scrollable content box.
  • Use CSS transitions over Javascript ones.
  • Disable visualisers or other Javascript-heavy pages.
  • Design for a high degree of colour and brightness variation between TV sets.
  • AppleTV uses a custom implementation of Safari, so may be buggy using newer features.

helpful code

The global Javascript variable IS_APPLE_TV is helpful when displaying different content on the AppleTV, but try using the following function so you can perform a basic functionality test for iTunes and AppleTV within Safari:

//after initialisation, TuneKit creates the window.iTunes object within Safari, which is why the IS_ATV variable is created on init and referenced later. You could just use this global variable instead of the isAppleTV() function if you wanted to
var IS_ATV = isAppleTV_init();
function isAppleTV_init() {
	//safari
	if(window.iTunes == undefined)
		return false;

	//itunes
	if(window.iTunes.platform == "Windows"
		|| window.iTunes.platform == "Mac"
		|| window.iTunes.platform == "Emulator")
		return false;

	return true;
}
function isAppleTV() {
	return IS_ATV;
}

Sometimes you need to manually control how the AppleTV remote keypresses will affect your project. The following code will allow you to override the default behaviour outlined by TuneKit:

var KEY_BACK = 8;
var KEY_ENTER = 13;
var KEY_LEFT = 37;
var KEY_UP = 38;
var KEY_RIGHT = 39;
var KEY_DOWN = 40;

function onKeyDown() {
	if(event.keyCode == KEY_BACK) {
		//your action
	} else if(event.keyCode == KEY_ENTER) {
		//your action
	} else if(event.keyCode == KEY_DOWN) {
		//your action
	} else if(event.keyCode == KEY_LEFT) {
		//your action
	} else if(event.keyCode == KEY_RIGHT) {
		//your action
	} else if(event.keyCode == KEY_UP) {
		//your action
	}

	//prevent default behaviour
	event.stopPropagation();
	event.preventDefault();
}

//assign key down listener
function onKeyDown_init() {
	window.addEventListener("keydown", onKeyDown, true);
}

Actionscript 3 draggable objects and kinematics example

Posted January 22nd, 2010 in Uncategorized by pixelbyter

I wrote this example a couple of years ago while researching Actionscript 3, inverse kinematics and the mathematic principles behind motion. I’m currently porting it to a CSS3 example.


Get the Flash source files here.

The following Actionscript 3 snippet is the bread and butter of the code that makes the above example possible:

//calculates angle of centre
var dx:Number = mouseX - thisPhoto.x;
var dy:Number = mouseY - thisPhoto.y;
var angle:Number = Math.atan2(dy, dx);

//handles spinning
var newRotation:Number = (angle - thisPhoto.origAngle) * 180 / Math.PI;

//handles dragging
var newX:Number = mouseX - Math.cos(angle) * thisPhoto.distanceToCentre;
var newY:Number = mouseY - Math.sin(angle) * thisPhoto.distanceToCentre;

//sets new positioning
thisPhoto.x = newX;
thisPhoto.y = newY;
thisPhoto.rotation = newRotation;

mysqldump database backup for mixed InnoDB and MyISAM tables

Posted January 22nd, 2010 in Uncategorized by pixelbyter

Backing up a collection of MyISAM and InnoDB tables within a database via mysqldump can cause problems. MyISAM tables require locking, whereas InnoDB tables need to remain unlocked and can also take advantage of single transactions to speed up the backup process. I decided to write the following PHP script to list all of the databases and tables that the supplied user has permissions to view, which automatically uses the correct mysqldump settings for MyISAM and InnoDB table types, saving each table’s data to a separate SQL file. In the event of a connection error during the mysqldump of a table, the mysqldump command is repeated until it completes successfully.

$user = 'YOUR_USER';
$password = 'YOUR_PASSWORD';
$host = 'YOUR_HOST';
$backup_dir = '/path_to_backup_dir/'.date('YmdHis');

//mysqldump commands
$default_mysqldump_shared_command = 'mysqldump --force --user='.$user.' --password='.$password.' --host='.$host.' --quote-names --default-character-set=utf8 --verbose --compress';
$default_mysqldump_innodb_command = $default_mysqldump_shared_command.' --single-transaction --skip-add-locks --skip-lock-tables';
$default_mysqldump_myisam_command = $default_mysqldump_shared_command.' --opt';

//connect
set_time_limit(0);
if(!mysql_connect($host, $user, $password))
	exit;

//create backup dir
shell_exec('mkdir '.$backup_dir);

//cycle through databases
$query = "SHOW DATABASES";
$database_result = mysql_query($query);
while($database_data = mysql_fetch_array($database_result, MYSQL_ASSOC)) {
	//current db
	$current_database = $database_data['Database'];

	//create dir
	echo shell_exec('mkdir '.$backup_dir.'/'.$current_database);

	//get table list
	mysql_select_db($current_database);
	$query = "SHOW FULL TABLES WHERE Table_Type != 'VIEW'";
	$table_result = mysql_query($query);
	while($table_data = mysql_fetch_array($table_result, MYSQL_ASSOC)) {
		//current table
		$current_table = $table_data['Tables_in_'.$current_database];

		//get table type
		$query = "SHOW TABLE STATUS LIKE '{$current_table}'";
		$table_properties_result = mysql_query($query);
		$table_properties = mysql_fetch_array($table_properties_result, MYSQL_ASSOC);

		echo 'backing up '.$current_database.'.'.$current_table."\r\n";
		$success = false;
		while($success == false) {
			//innodb or not?
			if($table_properties['Engine'] == 'InnoDB')
				$result = shell_exec($default_mysqldump_innodb_command.' '.$current_database.' '.$current_table.'| gzip > '.$backup_dir.'/'.$current_database.'/'.$current_table.'.gz')."\r\n";
			else
				$result = shell_exec($default_mysqldump_myisam_command.' '.$current_database.' '.$current_table.'| gzip > '.$backup_dir.'/'.$current_database.'/'.$current_table.'.gz')."\r\n";

			//error
			if(strpos($result, 'Error 2013') !== false)
				echo 'lost connection, trying again'."\r\n";
			else
				$success = true;
		}
	}
}

iTunes LP/Extras Javascript redirects

Posted January 21st, 2010 in Uncategorized by pixelbyter

For iTunes LP and iTunes Extras content programatically loading a link is very difficult. Both iTunes and AppleTV don’t support the Javascript function windows.open() or the property window.location. I’ve managed to come up with this bypass (aka hack) which is demonstrated in the Javascript code below. It triggers a mouse click event emulating the user manually clicking a link on the page.

//click a link with id 'my_link'
clickLink(document.getElementById('my_link'));

function clickLink(elementId) {
	var elm = document.getElementById(elementId);

	//set up mouse click event
	var evt = document.createEvent('MouseEvents');
	evt.initMouseEvent('click', true, true, document.defaultView, 1, 0, 0, 0, 0, false, false, false, false, 0,  null);

	//trigger click event on html element
	elm.dispatchEvent(evt);
}