FFMPEG PHP installation on windows

We recently had a request from a client to install FFMPEG and the associated PHP module on a Windows server that was running Plesk 8.2 with PHP 5.2. This posed a bit of a problem because the FFMPEG project only releases source code releases, relying on others to make binary distributions. For linux distributions, it's usually in the package repository, but for windows, you have to dig around the internet until you can find someone that has compiled it for you, or you can compile it yourself. In addition, they refuse to go out of their way to make their code Windows friendly, which is fine, but it makes the process to compile it kind of complicated, since it doesn't work in Microsoft's C/C++ compiler by default, so you have to configure MinGW to compile it.

Finding a windows distribution of the main FFMPEG project was fairly easy. This site distributes binaries and also provides a very good guide on compiling it for yourself.

However, the PHP module is much harder. Especially since you have to find a module that is compatible with your version of PHP. Most of the available instructions point to PHP 4 modules, if you need a PHP 4 module, here is one good set of instructions. After much digging, I finally found a PHP 5 module here.

UPDATED: This link is now broken, we're currently hosting the PHP 5 module on our website here.

A complete set of instructions for completing this installation is below. My environment was Windows 2003, IIS 6, Plesk 8.2 and PHP 5.2.6:

• If you plan to use the FFMPEG command-line tool, download the binary package from arrozcru.org. Use a tool such as 7-zip to decompress it and expand the tar file where you want to place it.
• Download the zip file that includes the ffmpeg-php extension here and uncompress it. It includes several other software packages, you can ignore them, the files that we are interested in are in the ffmpeg-php-win32-all directory. UPDATED: This link is now broken, we're currently hosting the PHP 5 module on our website here.
  
• Copy the avcodec-51.dll, avformat-51.dll, avutil-49.dll and pthreadGC2.dll files to the c:\windows\system32 directory.
• Copy the php_ffmpeg.dll file to your PHP extensions directory. For me, this was "C:\Program Files\SWsoft\Plesk\Additional\PleskPHP5\ext"
• Edit your php.ini file and add the following bit. For me this was located at "C:\Program Files\SWsoft\Plesk\Additional\PleskPHP5\php.ini"
  
  extension=php_ffmpeg.dll
  
  
• If you are running PHP as an ISAPI filter, you'll probably need to restart IIS. I was running it as a CGI and I didn't need to restart.
  

Quick guide to UTF-8 in PHP regular expressions

There's a lot of different issues that come up with any sort of Unicode support in applications. Mainly because things get a lot more complicated than the standard 8-bit ASCII implementation. UTF-8 is the most common encoding in use for Unicode support. There have been many in-depth documents written about this and I encourage you to read them if you have to do anything more than modify a couple of lines of code. To modify regular expressions in PHP for UTF-8 characters, here are a couple of steps that can get you off the ground quickly:

* Use the preg functions, the ereg functions do not support Unicode and are going away in PHP 6.
* To enable unicode for a regex, add a /u flag. i.e.

if(preg_match("/[^[:space:]a-zA-Z0-9{1,}/u", $string)) {

* Find a good table of UTF-8 characters and their hexadecimal value, like this one. Use this table to look up the Unicode value, it should be in the format U+NNN where the NNN represents a variable length hexadecimal number. For example the euro symbol, € is U+20AC. To use this in a regular expression, use the \x{NNN} format, i.e. \x{20AC}.

* For example, to add € to the allowed characters in the above expression, use the following:

if(preg_match("/[^[:space:]a-zA-Z0-9\x{20AC}{1,}/u", $string)) {

Comparing email options for small/medium organizations

Often we'll have clients that need mail setup for their domain as almost an afterthought. The primary use of the domain is to serve a website to a global audience, and the e-mail addresses will primarily be used for customer support or communication among a small team. Usually, it's tacked on to the end of a list of requirements for server configuration, i.e. "oh yeah, and we'll need a mail server too." Generally "mail server" in this context means that they want SMTP, POP, IMAP, a webmail interface and an administrative interface that allows them manage their own user accounts and aliases. Unfortunately, this setup is a little more effort than just doing a "yum install mailserver", so this is generally what we recommend for mail server installation:

Don't do it.
Use a hosted solution instead, email is a headache to administer and is now standardized enough that it can be offered as a commodity. Standard edition Google Apps for business is free and meets most needs well, there are many other good solutions out there. However, this doesn't work for everyone, due to confidentiality concerns, customization needs, etc.

Zimbra
If you have the horsepower, run Zimbra. It's free, mostly open source and provides all of the functionality that I mentioned above along with several other features like calendaring. Unfortunately, it requires a lot of resources (especially memory) to run all of its separate components, so it usually needs its own server, which is overkill for a lot of <50 user installations.

Postfix + Dovecot + Squirrelmail + postfixadmin.
If you don't have the resources to run Zimbra and need to run an in-house mail server, this is probably your best bet for putting together a cost-effective system. Unfortunately, it requires more setup time than the other solutions, but you can easily run it on a low-powered server or in a virtual instance.

Qmail + Courier + Squirrelmail + qmailadmin
I'm not a big fan of qmail, but it does have a fair number of supporters and I support a couple of different installations, so I'll mention it. Similar to the Postfix bundle, it doesn't require many server resources, but can take a fair bit of setup time. However, qmail is generally more difficult to administer, less well supported, etc, so I would recommend using Postfix as your SMTP server.

PHP file upload best practices

We've had a couple of different scenarios dealing with PHP file upload issues lately. Out of this, I've come up with some guidelines for using the default PHP file upload mechanism.

The first thing to do is to make sure that you are dealing with file upload errors properly. If an error occurs, it will be reported in the $_FILES['examplefile']['error'] field and a list of the error codes are detailed here.

PHP Settings

The most common problem is dealing with large file uploads, both due to the size of the files and the time it takes to upload the files from slower connections. Relevant settings that will need to be adjusted for larger files are:

upload_max_filesize
post_max_size
memory_limit

In addition, you'll need to make sure that you have the following settings adjusted to allow enough time for uploads to occur:

max_execution_time
max_input_time

Another method of limiting the size of file uploads is to have a form input with the name of MAX_FILE_SIZE. PHP will cut off uploads that are larger than the value of this field.


Apache Settings

By default, most Apache configurations do not need to be adjusted for file uploads. However, if you're using the LimitRequestBody directive, make sure that it is large enough to accomodate the uploaded files.

IIS Settings

In addition to the PHP settings for time limits, you will probably need to adjust settings in IIS to make sure that your connections do not time out. The most relevant metabase settings are:

ConnectionTimeout (defaults to 120 seconds)
CGITimeout (defaults to 500 seconds)

More flash reverse engineering

The post that I wrote last week that detailed how to use Flare to take a look at the ActionScript for a compiled SWF object. This is useful when you are trying to figure out the details of a flash apps behavior, but what if you actually need to change something about the SWF? Well, the same guy that wrote Flare also wrote a tool call flasm that will disassemble a flash object to a human readable representation of the Flash byte code. It can also reassemble the byte code to a SWF file. This is not action script and unless you're willing to put in the time to become an expert on the flash virtual machine, it's probably not suitable for any major modifications. However, it's great for doing things like changing hard-coded constants.

The basic commands are flasm -d somefile.swf > somefile.flm for generating the assembly file, and flasm -a somefile.flm to generate the SWF file.

Here's an example of changing a hard-coded URL in a set of SWF files using flasm. flasm is cross-platform, and is available on windows, OS X and linux, these commands are from a linux system.


$ for file in `ls *swf`; do /path/to/flasm -d $file > ${file}.flm; done
$ perl -pi -e 's/http:\/\/www\.oldsite\.com\//https:\/\/www\.newsite\.com\//g' *flm
$ for file in `ls *flm`; do ~/flasm/flasm -a $file ; done

ApacheCon 2008: Java Monitoring and Troubleshooting

I'm finally getting around to watching the presentations from ApacheCon 2008. I plan to blog about them and highlight some of the best things about each session. It costs $99 for access to the presentations, but is more than worth it if you use Apache, Tomcat, Java or any other Apache project in a production environment.

The first one that I watched was Java Monitoring and Troubleshooting, since I've spent a lot of time in the past year chasing down bugs in Java apps. Almost every time I've thought "there's got to be an easier way to do this", now, thanks to Bill Au's presentation, I know some of the easier ways. The materials from his presentation are available here.

The entire presentation is great, but here were some of the highlights for me:

* I've used thread dumps generated by doing "kill -3" on java processes before, but although they contain useful information, I usually spend a lot of time digging for it. He highlights a couple of tools that he uses to analyze thread dumps more efficiently. Basically his methodology is to take three consecutive thread dumps 5 seconds apart. Then he uses the samurai.jar tool in conjunction with his own script overview.pl to analyze the 3 separate files. The overview.pl script gives a good big picture look at how many threads are in each state (runnable, locked, etc.) The samurai tool takes multiple thread dumps and shows how each thread changes state from dump to dump to give a historical perspective. His techniques are especially useful if you're looking for deadlocks or hung threads. Both of these tools are available along with the presentation material.

* His demonstration of how to compare the performance of separate garbage collection methods for your application. He starts by getting a log of garbage collection activity for your application for a reasonably long period (an hour or so) when it is under at least moderate use. To do this, use the -Xloggc: option on your java command line. Get a log file for each GC method that you want to compare. Then download the HPjmeter tool released by Hewlett Packard. Then you can open up the logs from HPjmeter and get all sorts of statistics, including how much overhead, the number of full garbage collection events, and average time for each garbage collection. And to make things even better, you can open multiple log files from different GC methods and graph them against each other to get an easy to read picture of the pros and cons of each.

* Another neat thing was his demo of jhat, the heap analysis tool released by Sun to view the objects in the heap. It's released in Java 6, but can also process heap dumps from 1.4 and 1.5. Once started, it provides a web interface to browse the heap and view objects. The interface make it really easy to track down objects that are taking up more than their fair share of space, but just in case you need more flexibility, there's Object Query Language (OQL). As the acronym suggests, it's a SQL-like interface to the heap. So you can issue statements like this one that finds all finalizable objects and the heap size used by each:

select { obj: f.referent, size: sum(map(reachables(f.referent), "sizeof(it)")) }
from java.lang.ref.Finalizer f
where f.referent != null


Neat stuff! Again, I encourage you to view the entire thing or at least read all of the presentation slides, because all of the information is really good.

Flash config files and MIME types

I ran into this situation for the first time today. A client had purchased a Flash application and couldn't get it to work correctly. The company they bought it from said that the configuration file wasn't getting loaded correctly, but couldn't provide any further guidance. The tech support rep couldn't provide a location of the config file or any details about how it was being loaded. Since the technical support was so lacking and I had no intention to steal their code, I figured that the only thing I could do was take a peek inside the SWF. I used Flare to decompile the Actionscript from the SWF and started looking to see how they were loading the configuration. First this tidbit:

function loadConfig() {
var v4 = new LoadVars();
v4.onLoad = function (success) {
....
};

v4.load(_url + '.cfg');
}

Hmm... So they were using the builtin LoadVars.load() method to retrieve the config file from a URL that ended in ".cfg" Looking a bit further, it turns out that the _url variable was defined as follows:

this._url = xmlx.config.requestURL;

So they're trying to load a file with the same url as the SWF file with a .cfg appended to the end of it. Punching that address into a browser gave a 404 error. After checking to make sure that the permissions were okay and that other files from that directory were accessible, I took a look at the MIME types. This was an IIS server and by default IIS doesn't serve up content without a defined MIME type. As soon as I added ".cfg" as "text/plain" and restarted IIS, everything worked fine.