Latest Entries »

In this part of the series I will cover how to create your first job in Jenkins which will make use of all of the tools installed in the previous two sections of this series.

If you haven’t seen the previous two sections, they can be reached via the links below.

Part 1: Installing the PHP Tools and Jenkins

Part 2: Installing the Plugins and Configuring Jenkins

To begin I will talk about the directories and files you will need to have in your repository for your build to be successful, followed by a discussion of each of those files list, and then then I will cover the creation of your first job.  But, first, I will provide the links to the downloads in this post, up front for convenience.

Downloads

For convenience, here are all of the downloads covered in this post, so you don’t need to ferret out the links.

  1. Project directory structure and files –> here
  2. Jenkins job template –> here

Directory Structure and Files

Below is the minimal directory structure and files that you will need for the build script to work in Jenkins.  StructureExample is the root folder for the repository (just think of it as the trunk folder in a subversion repository)

Screen Shot 2013-05-15 at 2.57.59 PM

Directory structure and minimal files

From the above image you can see that there are two files “build.xml” and “phpunit.xml” in the root directory.

  • build.xml is an XML file that tells Ant what to do when performing a build.
  • phpunit.xml defines several parameters for PHPUnit to use during its portion of the build.

The location of these files are pretty standard; however, what is not so standard is that I have split the unit tests out into separate directories under the tests directory.  This was done to keep all of the PHP Unit tests separate from the SimpleTest tests.

For convenience you download the above directory structure and files  here.  The download included several versions of build.xml (build.xml.both, build.xml.simpletest, build.xml.phpunit) so you can pick the appropriate version for your build by renaming the appropriate file to build.xml.

Build.xml

The build.xml file is the most important file in the directory; it tells ant what to do during execution.  The file is a little long, roughly 178 lines and created the directory structure shown below.  Since this file is so important I will cover the sections that will be used while performing a build on Jenkins.

Jenkins Build directory

 

The fist thing to do is to put the name of your project in the file, in the name attribute for this line:

<project name="your-project-name-here" default="build">

Note that the above line also sets the default target to perform “build”. This line;

<target name="build"
        depends="prepare,lint,phploc,pdepend,phpmd-ci,phpcs-ci,phpcpd,phpdoc,phpunit,simpletest,phpcb"/>

This target depends on several other targets to be successful (meaning each of the dependent targets need to complete successfully).

Dependent Targets:

  • prepare –> Ensures that the workspace is setup correctly by wiping out old folders and remaking them.
  • lint –> calls php -l, to perform a syntax check on all of the php files in the directory.
  • phploc –> Uses PHP Loc to determine the number of lines of code in the project.
  • pdepend –> Runs pDepend and creates the dependency reports.
  • phpmd-ci –> Runs PHP Mess Detector to check for code smells and generates some reports of the results.
  • phpcs-ci –> Runs PHP CodeSniffer on the code in the project to check for coding standard violations and generats a report of the results.
  • phpcpd –> Runs PHP Copy Paste Detector which looks for duplicate lines of code.
  • phpdoc –> Generates API documentation using PHP Documentor 2
  • phpunit –> Performs the tests located in tests/phpunit, and uses phpunit.xml for its configuration. After which it creates a report of the results
  • simpletest –> Performs the tests in the TestSuite defined in test/simpletest/TestSuite.php, and then creates a report of the results
  • phpcb –> Creates a browsable report of your code

Target Definitions

Prepare

The prepare target depends on the clean target, which removes all the directories under the build directory in the workspace. After which, the prepare target re-creates them. ${basedir} contains the path to the current projects (jobs) workspace. For example, if your job was called “example-job” is would contain the path /var/lib/jenkins/jobs/example-job/workspace.

<target name="clean" description="Cleanup build artifacts">
   <delete dir="${basedir}/build/api"/>
   <delete dir="${basedir}/build/code-browser"/>
   <delete dir="${basedir}/build/coverage"/>
   <delete dir="${basedir}/build/logs"/>
   <delete dir="${basedir}/build/pdepend"/>
</target>

<target name="prepare" depends="clean" description="Prepare for build">
    <mkdir dir="${basedir}/build/api"/>
    <mkdir dir="${basedir}/build/code-browser"/>
    <mkdir dir="${basedir}/build/coverage"/>
    <mkdir dir="${basedir}/build/logs"/>
    <mkdir dir="${basedir}/build/pdepend"/>
</target>

Lint

This target uses the php command with the -l parameter to perform a syntax check on the php files in the project. While excluding the tests directory from its check.

<target name="lint" description="Perform syntax check of sourcecode files">
    <apply executable="php" failonerror="true">
        <arg value="-l" />

        <fileset dir="${basedir}">
            <include name="**/*.php" />
            <!-- exclude the tests directory, repeat line below to exclude more -->
            <exclude name="tests/**" />
            <modified />
        </fileset>

        <fileset dir="${basedir}/tests">
            <include name="**/*.php" />
            <modified />
        </fileset>
    </apply>
</target>

Phploc

This uses PHP Loc to determine the size of the project. The constructed command looks like phploc –exclude ${basedir}/tests –log-csv ${basedir}/build/logs/phploc.csv ${basedir}.
This command tells phploc to look at the code in ${basedir} but exclude the tests directory and to publish the report to build/logs/phploc.csv

<target name="phploc" description="Measure project size using PHPLOC">
    <exec executable="phploc">
        <!-- exclude tests directory -->
        <arg value="--exclude" />
        <arg path="${basedir}/tests" />
        <arg value="--log-csv" />
        <arg value="${basedir}/build/logs/phploc.csv" />
        <arg path="${basedir}" />
    </exec>
</target>

Pdepend

This target uses PHP Depend to create several metric reports about your code.

  1. An Overview pyramid covering inheritance, size and complexity, and coupling. You can find information about the pyramid here
  2. An Abstract Instability Chart, which you can find information on here

The target, again ignores the tests directory, as most of the targets do.

<target name="pdepend" description="Calculate software metrics using PHP_Depend">
    <exec executable="pdepend">
        <arg value="--ignore=${basedir}/tests,${basedir}/docs" />
        <arg value="--jdepend-xml=${basedir}/build/logs/jdepend.xml" />
        <arg value="--jdepend-chart=${basedir}/build/pdepend/dependencies.svg" />
        <arg value="--overview-pyramid=${basedir}/build/pdepend/overview-pyramid.svg" />
        <arg path="${basedir}" />
    </exec>
</target>

Phpmd-ci

This target uses PHP Mess Detector to check your code for common smells (indicators of possible bad coding or design). There are several rulesets you can use when checking your code: codesize, controversial, design, naming, and unusedcode. You can read about the rulesets here to figure out what each one covers.

You can also define your own rulesets of modify existing rulesets using an xml file. To learn more about there go here and checkout the menu on the right.

The current target configuration tells phpmd to create an xml report, using the codesize ruleset, and place the report in build/logs/pmd.xml. It should examine the current workspace and exclude the test directory.

<target name="phpmd-ci" description="Perform project mess detection using PHPMD creating a log file for the continuous integration server">
    <exec executable="phpmd">
        <arg path="${basedir}" />
        <arg value="xml" />
        <arg value="codesize" />
        <arg value="--reportfile" />
        <arg value="${basedir}/build/logs/pmd.xml" />
        <arg value="--exclude" />
        <arg value="${basedir}/tests" />
    </exec>
</target>

Phpcs-ci

This target uses PHP CodeSniffer to check the code in the project, excluding the tests directory, for Zend coding standard violations. The results of the analysis are then put into a checkstyle report in build/logs/checkstyle.xml.

To learn about the available standards and functionality of PHP CodeSniffer, visit here

<target name="phpcs-ci"
        description="Find coding standard violations using PHP_CodeSniffer creating a log file for the continuous integration server">
    <exec executable="phpcs" output="/dev/null">
        <arg value="--report=checkstyle" />
        <arg value="--report-file=${basedir}/build/logs/checkstyle.xml" />
        <arg value="--standard=Zend" />
        <arg value="--ignore=${basedir}/tests" />
        <arg value="--extensions=php" />
        <arg path="${basedir}" />
    </exec>
</target>

Phpcpd

This target uses PHP Copy Paste Detecter to check for duplicate lines of code in your project, excluding the tests directory, and outputting the results in build/logs/pmd-cpd.xml

<target name="phpcpd" description="Find duplicate code using PHPCPD">
    <exec executable="phpcpd">
        <arg value="--log-pmd" />
        <arg value="${basedir}/build/logs/pmd-cpd.xml" />
        <arg value="--exclude" />
        <arg path="${basedir}/tests" />
        <arg path="${basedir}" />
    </exec>
</target>

Phpdoc

This target uses PHP Documentor 2 to create API documentation of your code. You can find documentation about using phpdoc here.

<target name="phpdoc" description="Generate API documentation using phpDocumentor 2">
    <exec dir="${basedir}" executable="phpdoc" failonerror="true">
        <!-- excluded directories need the / at the end, else phpdoc ignores it -->
        <arg line="--title='test title' -d . -t build/api -i tests/ " />
    </exec>
</target>

Phpunit

This target runs your unit tests using PHP Unit. By default, PHP Unit checks the current directory for a configuration xml file, which is our phpunit.xml file. Because we took this approach, there is not much in the definition of this target.

However, you can find plenty about what you can put in the configuration file here

<target name="phpunit" description="Run unit tests with PHPUnit">
    <exec executable="phpunit" failonerror="true"/>
</target>

Simpletest

This target defines how to perform our unit/functionality tests using SimpleTest. You’ll notice that this target definition is a little different than the others. The reason is, is that SimpleTest does not have a nice command line tool that takes parameters like the other tools we are using. To make SimpleTest work, we need to create a little inline bash script.

The one-liner bash script that we will use tells the shell to use php to run TestSuite.php file and then send its output to XML Lint for formatting and output into build/logs/simpletest-results.xml

<!-- Inline bash script to run TestSuite.php and pipe its results to xmllint -->
<!-- XML Lint then outputs the results to simpletest-results.xml -->
<target name="simpletest" description="Run SimpleTest unit tests">
    <exec dir="${basedir}" executable="bash" failonerror="true">
        <arg value="-c"/>
        <arg line='"php tests/simpletest/TestSuite.php | xmllint --format -o build/logs/simpletest-results.xml -"' />
    </exec>
</target>

Phpcb

This target uses PHP CodeBrowser to create a browsable interface of your code in your project.

<target name="phpcb" description="Aggregate tool output with PHP_CodeBrowser">
    <exec executable="phpcb">
       <arg value="--log" />
       <arg path="${basedir}/build/logs" />
       <arg value="--ignore" />
       <arg path="${basedir}/tests" />
       <arg value="--source" />
       <arg path="${basedir}" />
       <arg value="--output" />
       <arg path="${basedir}/build/code-browser" />
    </exec>
</target>

phpunit.xml

This file is used to store our configuration for running PHP Unit.

For the over definition, we tell php unit to not backup and restore $GLOBALS or static attributes of user defined classes, to run in strict mode and have verbose output.

For our test suite settings, we set the name (which you should change) and tell php unit to run all of the tests in the php files in tests/phpunit.

Next for our logging settings, we tell PHP Unit to create a coverage report in build/coverage, with the title of your project (don’t forget to set this attribute value) and several other attributes. We also want a coverage-clover report in build/logs/clover.xml and a junit report in build/logs/junit.xml

To learn creating your own PHP Unit configuration file visit here

<?xml version="1.0" encoding="UTF-8"?>

<phpunit backupGlobals="false"
         backupStaticAttributes="false"
         strict="true"
         verbose="true">

  <testsuites>
    <testsuite name="Your Test Suit Name Here">
      <directory suffix='.php'>./tests/phpunit</directory>
    </testsuite>
  </testsuites>

  <logging>
    <log type="coverage-html" target="build/coverage" title="Your Project Name Here"
         charset="UTF-8" yui="true" highlight="true"
         lowUpperBound="35" highLowerBound="70"/>
    <log type="coverage-clover" target="build/logs/clover.xml"/>
    <log type="junit" target="build/logs/junit.xml" logIncompleteSkipped="false"/>
  </logging>

</phpunit>

TestSuite.php

This file defines out Test Suite for SimpleTest. For our test suite, we define the class MyTests and add the Test Files we want to run using the addFile method. Take note of the use of the magic word __DIR__. This is used so no matter where this file is being invoked from, we won’t have to worry about how the test files are included.

To run the tests in the test suite, a new instance of the class is made and then ran using the JUnitXMLReporter class, so we can jUnit and xUnit compatible output. I had noticed in some other blogs that there was a problem with the header and footer printing in non xml; however, this approach does not have that problem.

/**
 * Include the simpletest file so we can extend TestSuite
 */
require_once 'simpletest/simpletest.php';
/**
 * Include the JUnitXMLReporter class so the output is jUnit and xUnit compatible
 */
require_once 'simpletest/extensions/junit_xml_reporter.php';

class MyTests extends TestSuite
{
    function __construct()
    {
        parent::__construct();

        //add the files containing the tests we want to perform
        //note the use of __DIR__ so we don't have to worry about file location
        //when running the tests in Jenkins
        $this->addFile(__DIR__.'/TemplateTest.php');
    }

}

$test = new MyTests();
$test->run(new JUnitXMLReporter());

Example Test File – TemplateTest.php

I thought it might be pertinent to show an example test file for use with the Test Suite defined above. In the code below, you will notice that I never include simpletest/autorun.php, I only include simpletest/web_tester.php. This is because I want complete control over the running of the tests and their output.

<?php
require_once 'simpletest/web_tester.php';

class TemplateTest extends WebTestCase {
    function testLoginPageShowForm() {
        $this->get('https://example.domain.com/site/login/form.php');
	$this->assertFieldById('_userName');
    }
}

StackTest.php

This is a simple PHP Unit test file that just does some pushing and popping of an array. In fact, it is the example from chapter 4 “Writing Tests for PHPUnit” from PHP Units web site here

<?php
/**
 * Include the PHPUnit Framwork so we can extend the TestCase class
 */
require_once 'PHPUnit/Autoload.php';

class StackTest extends PHPUnit_Framework_TestCase
{
    /**
     * Tests pushing and popping of elements on and off of an array
     *
     * @return Void
     */
    public function testPushAndPop()
    {
        $stack = array();
        $this->assertEquals(0, count($stack));

        array_push($stack, 'foo');
        $this->assertEquals('foo', $stack[count($stack)-1]);
        $this->assertEquals(1, count($stack));

        $this->assertEquals('foo', array_pop($stack));
        $this->assertEquals(0, count($stack));
    }
}

Setting Up Your First Jenkins Job

This section will cover setting up your first job in jenkins using an existing template. The first thing you will need to do is add the job template to Jenkins.

Adding the job-template To Jenkins

There is a lot of configuration that needs to be done to use all the tools that we’ve installed and referenced in our build.xml file. So, let’s cut out a lot of the work and use an existing job template that has the majority of work done for us already.

  1. Go to the Jenkins job directory
    $cd /var/lib/jenkins/job
  2. Download the job template
    $wget https://googledrive.com/host/0B8dI-j7Xg0phVDRjMnN6QUhfcXc -O job-template.tar.gz
  3. Extract the job template
    $tar xvf job-template.tar.gz
  4. Set the permissions on the directory
    $chown -R jenkins:nogroup job-template/

Setting up the Jenkins Job

  1. Login to Jenkins and click “New Job”
  2. Enter the name of your job

    NOTE: It is a good idea to not have spaces in your job name as it can cause problems

  3. Select “Copy Existing Job”
  4. In the “Copy From” field, type “job-template” (without the quotes)Your screen should look like this

    Jenkins - New Job Screen

  5. Click “OK”
  6. Uncheck “Disable Build”
  7. If you want to use project based security.
    Check “Enable project-based security”You’ll see a permission matrix appear and a field to add a user or group to the matrix

    Jenkins - Project Based Security

    Enter the user or group you want to set the rights for into the “User/group to add” field and click add
    Then set their permissions

    NOTE: Make sure to add yourself or you’ll encounter an issue where your build history disappears until you restart Jenkins

  8. Under the “Source Code Management” section, select “Subversion
  9. Enter the URL to your subversion repository in the “Repository URL” field

    Jenkins - Subversion Settings
    If Jenkins cannot talk to your repository, you may need to enter your credentials.

    1. Click the “enter credentials” link in the error message
    2. Select Username/password authentication
    3. Enter your credentials
    4. Click OK

      Jenkins - Subversion Authentication

  10. Under the “Build Triggers” sectionSelect Poll SCM and use the document linked in the button to determine what you need to enter for your schedule
  11. If you do not want to use PHP Unit or SimpleTest
    Way down under the “Publish xUnit test result report” section, click the delete button for for the unit test suite you don’t want to use.
    PHP Unit

    Jenkins - PHP Unit Settings

    SimpleTest
    Jenkins - SimpleTest Settings

  12. Click Save
    Note: If you click Apply and then Save, Jenkins throws a parse error
  13. With your job created, you can now perform a build by clicking “Build” on the left side of the screen

As a final note, you will not see many graphs on the main page of a job until you have performed at least two builds.

In the first part of this series I covered installing several PHP tools for continuous integration testing (PHPUnit, Mess Detecter, Copy Paste Detecter, Code Sniffer, Code Coverage, Documenter 2, Lines of Code, and Simple Test) and installing Jenkins.  You can find the first part here  In this part I will cover installing the needed plugins to use the installed PHP tools and to authenticate to active directory, and to use HTTPS (SSL) instead of the standard HTTP connection.

If the Jenkins server is currently running (if you just finished up part 1, it probably is) stop the server by running the following command

$/etc/init.d/jenkins stop

Setup the LDAP / AD Certificate

If you care planning to connect to an LDAP server or Active Directory and use LDAPS when doing so, you will need to let Jenkins know about the certificate the server has. To do so following the following steps:

Download the InstallCert tool from here

Install the unzip application

$apt-get install unzip

Unzip the tool

$unzip InstallCert.zip

Move the tool the the Java bin location

$mv InstallCert* $JAVA_HOME/jre/bin

Get the servers certificate. If you have multiple servers to get certs from, repeat this step for each.

$java InstallCert someServer.example.com:636

If you are prompted for anything, just press enter to continue on.

The certificate(s) will be placed in a file called jssecerts. Now we need to import this file into the cacerts file that java uses.

$keytool --importkeystore -srckeystore jssecacerts -destkeystore $JAVA_HOME/jre/lib/security/cacerts -noprompt

The password for the keystore is “changeit”.

To inform Jenkins of the cacerts file, edit the file /etc/defaults/jenkins and add the line to the file before the end (preferable under the commented out JAVA_ARGS line)

JAVA_ARGS="-Djavax.net.ssl.trustStore=/etc/ssl/certs/java/cacerts"

Setting Up SSL (HTTPS)

Edit the file /etc/defaults/jenkins and add the following lines

HTTPS_PORT=8443

HTTPS_KEYSTORE=/etc/ssl/certs/java/YourCertFile.crt
HTTPS_KEYSTORE_KEY=/etc/ssl/certs/java/YourCertKeyFile.key

Also set HTTP_PORT line to -1 to disable it

HTTP_PORT = -1

At the bottom of the file, set the JENKINS_ARGS line to the following

JENKINS_ARGS="--webroot=/var/cache/jenkins/war --httpPort=$HTTP_PORT --ajp13Port=$AJP_PORT --httpsPort=$HTTPS_PORT --httpsCertificate=$HTTPS_KEYSTORE --httpsPrivateKey=$HTTPS_KEYSTORE_KEY"

Installing the Plugins

  1. Open a web browser and go to https://yourServer. example.com:8443
  2. Click on Manage Jenkins
  3. Click on Manage Plugins
  4. Select the “Available” tab
  5. Select the following plugins
    • Active Directory Plugin
    • checkstyle
    • clover php plugin
    • dry
    • html publisher plugin
    • jdepend plugin
    • plot plugin
    • pmd plugin
    • violations
    • xUnit plugin
  6. Click “Install without Restart”

Setup Authentication to Active Directory

  1. Go to Manage Jenkins -> Configure Global Security
  2. Check “Enable Security”
  3. Select “Active Directory” under the Security Realm section
  4. Click “Advanced”
  5. Set the following fields:Domain Name: example.com
    Bind DN: CN=someUser,OU=Users,DC=example,DC=com
    Bind Password: TheBindUsersPassword
  6. Click “Test”
  7. If all is good, click “Apply”, else trouble shoot the issue
  8. Click “Save”
  9. Try to authenticate as a known user by clicking “log in” in the upper right corder and authenticating

NOTE: This setup did not work when I entered a domain controller. If I left the field blank, Jenkins was able to find the appropriate server and authenticate without issue.

Setting up LDAP Authentication

If you don’t want to use the active directory plugin, you can also authenticate using LDAP functionality Jenkins already has.

  1. Go to Manage Jenkins -> Configure Global Security
  2. Check “Enable Security”
  3. Under the Security Realm section, select “LDAP”
  4. Set the following fields:Server: ldaps://someServer.example.com:636
    User Search Filter: sAMAccountName={0}
    Manager DN:CN=someUser,OU=Users,DN=example,DN=com
    Manager Password:The manager users password
  5. Click “Apply”
  6. Click “Save”
  7. Test authenticating as a known user by clicking the “log in” link in the upper right corner and trying to log in

NOTE: If you intend to authenticate to an LDAP server like eDir (Novell) do not set the User Search Field to sAMAccountName={0} as it will not work

A Little Extra Security

To help prevent Cross-Site Scripting do the following:

  1. Go to Manage Jenkins -> Configure Global Security
  2. Check “Prevent Cross Site Request Forgery Exploits”
  3. Select “Default Crumb Issuer”
  4. Click “Apply”
  5. Click “Save”

Configuring Access

Now that you have users authenticating to Jenkins, you should limit what they can do. By default, Jenkins allows all users to do all things.

  1. Go to Manage Jenkins -> Configure Global SecurityUnder the “Authorization” section select “Project-based Matrix Authorization Strategy”
  2. Enter your username in the “User/group to add” field and click the “Add” button
  3. You should probably give yourself full permissions, you can do this quickly by clicking the image next to the red X on the right side of the row for your user
  4. If you want to add a group, just enter the group name in the “User/group to add” filed and click add. You used to have to prefix groups with ROLE_, but this is no longer required
  5. Set the permissions for the group or users you add to the list

NOTE: Under this authorization scheme, the permissions given to the users or groups here should be their base permissions site wide. In other words, give them the minimum amount here. Then in the projects they are a working on, you can specify additional rights under the “job configuration screen” for the project.

This is part one of a multi part series where I intend to cover the installation of Jenkins and several PHP tools to use for continuous integration. Along with the configuration of Jenkins and setting up a project and the build xml. This tutorial covers all of these steps in an Ubuntu 12.04 environment.

In this part of the series I will cover installing the following software and tools:

  • Ant
  • Pear
  • PHP 5.4
  • PHP Code Browser
  • PHP Code Coverage
  • PHP Code Sniffer
  • PHP Copy Past Detector
  • PHP Documenter 2
  • PHP Lines of Code
  • PHP Mess Detector
  • PHP Unit
  • Simple Test
  • XML Lint
  • XSL
  • XDebug

Installing PHP 5.4 and Pear

Ubuntu 12.04 by default installs PHP 5.3, so we will need the PPA repository where PHP 5.4 is available from. To do this, we will use the add-apt-repository tool. First we will need to install this tool and then add the repository.

Adding the tool

$apt-get install python-software-properties

Adding the repo

$add-apt-repository ppa:ondrej/php5

Installing PHP 5.4, PHP5.4 dev tools, and pear

$apt-get install php-pear php5-cli php5-common php5 php5-dev

Installing Ant, XSL, XML Lint, and XDebug

Ant will be used to run our build script later.  XSL and XDebug will be used by PHPUnit.  XML Lint will be used to format our reports created by SimpleTest.

To perform the installation of these tools, run the following command.

$apt-get install ant php5-xsl php5-xdebug libxml2-utils

Installing the PHP Tools

Setting up Auto Channel Discovery

$pear config-set auto_discover 1

PHP Code Browser

$pear channel-discover pear.phpqatools.org
$pear install phpqatools/PHP_CodeBrowser
$pear install pear/Text_Highlighter-beta

PHP Code Coverage

$pear install pear.phpunit.de/PHP_CodeCoverage

PHP Code Sniffer

$pear install --alldeps PHP_CodeSniffer

PHP Copy Paste Detecter

PHP-CPD requires the ConsoleTools from ezComponents, so let us take care of that first

$pear channel-discover components.ez.no
$pear install ezc/ConsoleTools

Now we can install PHP-CPD

$pear install --alldeps pear.phpunit.de/phpcpd

PHP Documenter 2

To use the inheritance chart feature, you’ll need to install GraphViz like so (this is not needed, but it’s nice)

$apt-get install graphviz

Now we can install PHP Documenter 2 after discovering its channel

$pear channel-discover pear.phpdoc.org
$pear install --alldeps phpdoc/phpDocumentor-alpha

PHP Lines of Code

$pear install --alldeps pear.phpunit.de/phploc

PHP Mess Detector

To install PHP-MD, we first need to install imagemagick and libmagickwand-dev to take care of the dependencies for imagick.

Installing imagemagick and libmagickwand-dev

$sudo apt-get install imagemagick libmagickwand-dev

Currently the stable version of imagick does not support PHP 5.4, so we need to grab the RC version by doing the following

$pecl config-set preferred_state beta
$pecl install imagick
$pecl config-set preferred_state stable

Now we can install PHP-MD after having pear discover the channels it needs (the auto discover setting didn’t work for this install, so we need to hold pears hand)

$pear channel-discover pear.phpmd.org
$pear channel-discover pear.pdepend.org

$pear install --alldeps phpmd/PHP_PMD

PHP Unit

$pear install --alldeps pear.phpunit.de/PHPUnit

Simple Test

Simple test is an easy to use functional testing library some find easier to use than PHPUnit. To install this library, fist download it like so

$wgetc -c http://sourceforge.net/projects/simpletest/files/simpletest/simpletest_1.1/simpletest_1.1.0.tar.gz

Then move the tar file to the place you want to store the library (I prefer /opt>

$mv simpletest_1.1.0.tar.gz /opt

Uncompress the tar

$cd /opt
$tar xvf simpletest_1.1.0.tar.gz

Both JUnit and xUnit use XML reports to publish test results and SimpleTest does have the ability to output its results in XML, but the format is not compatible with jUnit or xUnit.
To get the output in the format we need, we will need to use the JUnitXMLReporter class. This class is not in the download by default and you will have to download it separately.

Go to the extensions directory

$cd simpletest/extensions

Download the JUnitXMLReporter class

$wget --output-document=junit_xml_reporter.php http://simpletest.svn.sourceforge.net/viewvc/simpletest/simpletest/tags/1.1.0/extensions/junit_xml_reporter.php?revision=2050

Set permissions for the simplest directory

$chown -R jenkins:root /opt/simpletest
$chmod -R 775 /opt/simpletest

Add the library to the include path

$vim /etc/php5/cli/php.ini

Add “:/opt” to the php_include line like so

include_path = ".:/usr/share/php:/opt"

Save the file (escape, :wq, enter)

Installing Jenkins

To install Jenkins we first need to setup OpenJDK 7 as it is recommended over Open JDK 6, which Ubuntu will try to install.

Installing OpenJDK 7

Installing OpenJDK7 JDK

$apt-get install openjdk7-jdk

The install will not setup the $JAVA_HOME environment variable, so let’s take care of that.

Edit /etc/environment

Add the following line

JAVA_HOME="/usr/lib/jvm/java-7-openjdk-amd64/"

Note: If your install is not a 64-bit install your path will be different, please check /usr/lib/jvm to determine your proper path

Not that java is installed and the $JAVA_HOME variable has been defined, please exit (exit sudo or logout) and restart your terminal session (re login or enter sudo again)

Installing Jenkins

To perform the install, we first need to tell apt-get to get the latest versions from jenkins-ci.org

Run the following command to the key for jenkins-ci.org’s repo

wget -q -O - http://pkg.jenkins-ci.org/debian/jenkins-ci.org.key | sudo apt-key add -

Add the repo to apt-gets’ sources

sh -c 'echo deb http://pkg.jenkins-ci.org/debian binary/ > /etc/apt/sources.list.d/jenkins.list'

Update apt-get

apt-get update

Install Jenkins

$apt-get install jenkins

In The Next Part…

In the next part of the series I will cover installing the plugins we will use in Jenkins, configuring Jenkins to use SSL, and configuring Jenkins to authenticate to Active Directory

For this example, let’s say you have a site you want to post data to so a newly trained employee is added to a mailing list.  In the older days of php, you could do this with http_post_params; however, this function is now deprecated.  Instead, you can use the curl method, or you can use some Zend libraries.

Since this is a post on using Zend, I think you know which one I chose to do.

The following example is a really simple one.  I’m just using a configuration file to hold my options and parameters to use for creating a curl request.  Then I have a script that reads the config  file, creates the request, sends the request, and then checks the response.

The Configuration File

; --configuration file /configFile/myConfig.ini
[production]

;connection info
 curl.mailingList.adapter = Zend_Http_Client_Adapter_Curl
 curl.mailingList.uri = "http://www.example.com/mailinglist/api/add/"

;authentication/post parameters
 curl.mailingList.postParameter.APIUSER = "admin_user"
 curl.mailingList.postParameter.APIPWD = 'some_password'

;configuration options
 curl.mailingList.options.curloptions[CURLOPT_RETURNTRANSFER] = true
 curl.mailingList.options.curloptions[CURLOPT_HEADER] = false
 curl.mailingList.options.curloptions[CURLOPT_ENCODING] = ""
 curl.mailingList.options.curloptions[CURLOPT_AUTOREFERER] = true
 curl.mailingList.options.curloptions[CURLOPT_CONNECTTIMEOUT] = 120
 curl.mailingList.options.curloptions[CURLOPT_TIMEOUT] = 120
 curl.mailingList.options.curloptions[CURLOPT_MAXREDIRS] = 10
 curl.mailingList.options.curloptions[CURLOPT_POST] = 1
 curl.mailingList.options.curloptions[CURLOPT_VERBOSE] = 1
 curl.mailingList.options.timeout = 120
 curl.mailingList.options.maxredirects = 10

Performing the Curl

To actually perform the curl, we will need to load the configuration file into our code and then prepare the curl to send.

<?php
/**
 * Create a post request to add a new trainee to the mailing list
 *
 * PHP Version 5
 *
 * @created 10/12/2011
 * @updated 10/12/2011
 */

/**
 * Include the Zend_Loader class
 */
require_once 'Zend/Loader.php';

//load the needed Zend libraries
Zend_Loader::loadClass('Zend_Config_Ini');  //for reading the configuration file
Zend_Loader::loadClass('Zend_Http_Client_Adapter_Curl');
Zend_Loader::loadClass('Zend_Http_Client');

//read in the configuration file
$config = new Zend_Config_Ini('/configFile/myConfig.ini', 'production');

//setup our post params (subscribees is the name of the form field)
$postParam = array('subscribees' => 'joeExample@corporation.com');

//try to create and send the curl
 try
 {
        //setup the curl connection
        $adapter = new Zend_Http_Client_Adapter_Curl();
        $adapter->setConfig($config->curl->mailingList->options->toArray());

        //instantiate the http client and add set the adapter
        $client = new Zend_Http_Client($config->curl->mailingList->uri);
        $client->setAdapter($adapter);

        //add the post parameters from our config file (the authentication part)
        $client->setParameterPost(
            $config->curl->mailingList->postParameter->toArray()
        );

        //add our post parameters to the request
        $client->setParameterPost($postParam);

        //perform the post, and get the response
        $response = $client->request(Zend_Http_Client::POST);

        //check the response for success
        if (strpos($response, "<h5>Successfully subscribed:</h5>") !== false) {

            echo "<p>Added {$fullName} to trainee email list successfully.</p>";

        } else if (strpos($response, "Already a member") !== false) {

            echo "<p>Already a member of the trainee email list";

        } else {

             echo "<p style='color: red;'>ERROR: Not added to the trainee email list</p>";
        }
}
catch (Zend_Http_Client_Adapter_Curl_Exception $e)
{
    echo "<p style='color: red;'> ERROR: Failed to add the employee to the trainee email list.</p>";
}
catch (Zend_Http_Client_Exception $e)
{
     echo "<p style='color: red;'> ERROR: Failed to add the employee to the trainee email list.</p>";
}
?>

As you can see, the script is not very complex. But it could be expanded to handle logging of errors and other functionality.

Installing PHP 5.4 and PEAR

PHP5.4 is pretty easy to install on windows.  Although it doesn’t have a self installing binary like some of the previous versions, the install is pretty straight forward.

  1. Download PHP 5.4 from php.net here (you want to download the VC9 x86 thread safe zip)
  2. Extract the downloaded zip to c:\php
  3. In your favorite browser go to http://pear.php.net/go-pear.phar to download go-pear.phar
  4. Move go-pear.phar from your download directory to c:\php
  5. Rename the file php.ini-development or php.ini-production to php.ini in the directory c:\php
  6. Add php to the PATH environment variable, by appending ;c:\php to it
    note: If you’re not sure how to set the path variable in windows 7 this is a good guide.
  7. Open a command prompt with administrative permissions (go to, all programs –> accessories and right-click command prompt and select “run as administrator”)
  8. Go to the php directory
    cd c:\php
  9. Beginning the PEAR install
    php go-pear.phar
  10. Select system for the install type (it’s the default, so you can just press enter)
  11. Next you will be shown the paths that PEAR will use, you want to change number 11 to c:\php\pear.ini
    –>To do this type 11, press enter, then type c:\php\pear.ini
  12. After the install has finished run c:\php\PEAR_ENV.reg to finish the installation

Installing PHP Codsniffer

  1. Open a command prompt with administrative rights
  2. Go to the PHP install directory
    cd c:\php
  3. Install PHP Codesniffer
    pear install PHP_CodeSniffer

There are several options for setting up CI (continuous integration) for PHP.  Some of your options include Hudson (now owned by Oracle), Jenkins (the fork from Hudson), PHPUnderControl, and CruiseControl to name a few. While they each have their strengths, I wanted something that would be easy to use and have a company backing.  Through my searching I found TeamCity.

Team City is free to use for you first 20 build configurations (that is the only limit), after that you could make a new server to service your additional builds or actually pay for it.

This part of the two part series will cover the setup and configuration of TeamCity in the following sections below:

  1. Post setup
  2. Installing TeamCity
  3. Configuring TeamCity
  4. Configuring Apache to Redirect to Teamcity

Post Setup

This section will cover the various things that need to be setup on our server before we can install and configure TeamCity.

Apt-get Installs

To begin, we need to install several packages using apt-get (you could also use aptitude if you want).  These installs will cover PHP, Apache, subversion, and other libraries/applications.  To install each of these packages below simply type “sudo apt-get install <package-name>

  • ant
  • build-essential
  • libtcnative-1 (this installs the Apache APR module)
  • openssl
  • php-pear
  • php5
  • php5-cli
  • php5-curl
  • php5-dev
  • subversion

Installing Java JDK 6

Java no longer is included with Ubuntu out of the box and only OpenJDK is in the package repository.  This creates a problem as TeamCity doesn’t work with OpenJDK.  However, it will work with Java JDK 6 (update 32) at the time of this posts writing.  Fortunately, installing Java from Oracle is rather painless, but you will need sudo to do these commands.

  1. Download the latest  JDK 6 Update from here (I use the .bin files as they are the easiest to install, so grab one of those, either x86 or x64 depending on your architecture).
  2. Go to where you downloaded the the bin file and make it executable by doing the following
    chmod +x jdk-6u32-linux-x64.bin
  3. Run the binary.  This will extract the files into a directory like jdk1.6.0_32
    ./jdk-6u32-linux-x64.bin
  4. Create a directory to put java in
    mkdir /usr/lib/jvm
  5. Move the java directory to the new on and rename it to something nicer
    mv jdk1.6.0_32 /usr/lib/jvm/java-6
  6. Inform ubutnu of the new java install
    update-alternatives --install "/usr/bin/java" "java" "/usr/lib/jvm/java-6/bin/java" 1
    update-alternatives --install "/usr/bin/javac" "javac" "/usr/lib/jvm/java-6/bin/javac" 1
    update-alternatives --install "/usr/bin/javaws" "javaws" "/usr/lib/jvm/java-6/bin/javaws" 1
  7. For each of the lines above you will need to configure the default  to use by executing the following lines and choosing the path to your new java install from the given list.  Note that if this is your only install of java for this system, you can skip this step.
    update-alternatives --config java
    update-alternatives --config javac
    update-alternatives --config javaws
  8. Now we need to add the JAVA_HOME variable to our environment and update the PATH variable.   Edit /etc/environment and add the following line to the top
    JAVA_HOME="/usr/lib/jvm/java-6"
  9. Add the following lines to the end of /etc/profile
    JAVA_HOME="/usr/lib/jvm/jdk-6"
    PATH=$PATH:$JAVA_HOME/bin
    export JAVA_HOME
    export PATH
  10. Reconnect to your ssh session or restart your terminal to update the profile settings and test that java and javac are on the path by issuing the following commands
    java --version
    javac --version

    You  should get output for each of these commands.

Installing the Pear Tools for PHP CI

For this setup, TeamCity is going to be using a fair number of PHP tools in it ant script.  We will be using the following PHP tools

  • PHP Codesniffer
  • PHP Copy And Past Detector
  • PHP Dead Code Detector
  • PHP Depend
  • PHP Documentor 2
  • PHP Mess Detector
  • PHP Unit

As you can see there are quite a few to install; however, it is relatively painless to do so (again you will need sudo to do this).

  1. Update the pear channel
    pear channel-update pear.php.net
  2. Update Pear
    pear update pear
  3. Discover the needed pear channels
    pear channel-discover pear.phpdoc.org
    pear channel-discover bartlett.laurent-laville.org
    pear channel-discover pear.phpunit.de
    pear channel-discover components.ez.no
    pear channel-discover pear.symfony-project.com
    pear channel-discover pear.pdepend.org
    pear channel-discover pear.phpmd.org
  4. Install the pear modules
    pear install bartlett/PHP_CompatInfo
    pear install --alldeps phpunit/PHPUnit
    pear install --alldeps PHP_Codesniffer
    pear install pdepend/PHP_Depend-beta
    pear install phpmd/PHP_PMD
    pear install phpunit/phpcpd
    pear install phpunit/phpdcd-beta
    pear install phpdoc/phpDocumentor-alpha

Installing TeamCity

This section covers the download and installation of TeamCity and the changes we need to make before first launch.

Installation

  1. Go to the TeamCity website here and select your os and click download.  If you are using command line, you can use wget -c http://download.jetbrains.com/teamcity/TeamCity-7.0.2a.tar.gz
  2. Unpack your download
    tar -xvf TeamCity-7.0.2a.tar.gz
  3. Move your new directory to an appropriate place (I prefer the opt directory)
    mv TeamCity-7.0.2a /opt/TeamCity
  4. Set the permissions for the directory (we are going to run as www-data, not root)
    chown -R www-data /opt/TeamCity
  5. Create the following script called teamcity in /etc/init.d  (vim /etc/init.d/teamcity)
    #!/bin/sh
    # /etc/init.d/teamcity -  startup script for teamcity
    export TEAMCITY_DATA_PATH="/opt/TeamCity/.BuildServer"
    
    case $1 in
    start)
    start-stop-daemon --start  -c www-data --exec /opt/TeamCity/bin/teamcity-server.sh start
    ;;
    
    stop)
    start-stop-daemon --start -c www-data  --exec  /opt/TeamCity/bin/teamcity-server.sh stop
    ;;
    
    esac
    
    exit 0

    The above script is from Johannes Rudolph’s Blog and modified to fit my configuration

  6. Add the script to the startup routine
    update-rc.d teamcity defaults

Before First Launch

There is not much you can configure before the first launch of TeamCity; however, we can configure what port it is going to use.  For this installation I intend to authenticate off of Active Directory and I do not want those accounts going over the wire in plain text so SSL needs to be configured.

  1. Edit the server.xml
    vim /opt/TeamCity/conf/server.xml
  2. In the section for SSL connector add the following connector (you can find this section by searching for SSL)
    <Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true"
    maxthreads="150" scheme="https" secure="true"
    SSLCertificateFile="/path/to/cert/file.crt"
    SSLCertificateKeyFile="/path/to/cert/file.key"
    SSLCertificateChainFile="/path/to/cert/chain.crt" />

    If you have a self generated certificate you probably do not have a chain file, so you can leave that attribute out.

Creating the Data Directory

  1. Start the server
    /etc/init.d/teamcity start
  2. Open a web browser and navigate to https://yourServer.domain.com:8443 (if you don’t have a DNS entry for your machine go to https://localhost:8443 or https://yourServersIP:8443)
  3. You should see a screen saying this is the first run for the server and asking if you want to proceed.  Obviously we do, so click Proceed
  4. Next it will show you the EULA which you will need to accept
  5. Now you will see a form asking you to fill out the information for the admin user.  Stop here and shutdown the server
    /etc/init.d/teamcity stop

Configuring TeamCity

This section will cover connecting TeamCity to an external database (in this case PostgreSQL) and having TeamCity authenticate against LDAP

Connecting TeamCity To PostgreSQL

  1. Navigate to the place where JDBC database drivers are stored for TeamCity
    cd /opt/TeamCity/.BuildServer/lib/jdbc/
  2. Download the latest PostgreSQL JDBC driver from hereor use wget like so
    wget -c http://jdbc.postgresql.org/download/postgresql-9.1-901.jdbc4.jar
  3. Make www-data the owner of the file
    chown www-data postgresql-9.1-901.jdbc4.jar
  4. Make the file executable
    chmod +x postgresql-9.1-901.jdbc4.jar
  5. Connect to your PostgreSQL server and use the following SQL to create the TeamCity user account
    CREATE USER TeamCityUser WITH PASSWORD 'something _secret';
  6. Create a database and set it’s owner to the new user using the SQL statement below
    CREATE DATABASE TeamCityDB  OWNER TeamCityUser ENCODING 'UTF-8';
  7. Go to TeamCitiy’s config directory in the data directory
    cd /opt/TeamCity/.BuildServer/config
  8. Copy the template database file for PostgreSQL to make it the database config file to use
    cp database.postgresql.properties.xml database.properties
  9. Edit database.properties file and set the following lines
    connectionUrl=jdbc:postgresql://db.server.address.com:5432/database_name
    connectionProperties.user=your_db_account
    connectionProperties.password=your_db_account_password

    If you where connecting to a server at dbServer.example.edu and using database TeamCity with account appAccount and password P@455W0rd, it would look like this

    connectionUrl=jdbc:postgresql:// dbServer.example.edu:5432/TeamCity
    connectionProperties.user=appAccount
    connectionProperties.password=P@455W0rd

    Notice that there are not quotes.  The reason for this is TeamCity automatically quotes the values for you.  If you add quotes, it will fail authentication.

Authenticating to LDAP

Authenticating against LDAP is pretty easy; however, it gets a bit tricky when you want to use a secure connection (LDAP with SSL).  In the following sections I will show you how to install the SSL certificate from an Active Directory domain controller and how to configure TeamCity to use LDAPS.

Importing the LDAPS Certificate

Before we can authenticate using LDAPS, we have to import the certificate that the domain controller is using so Java will trust it.  Method 1 is pretty easy to do and relatively painless, but does require you to download an additional Java tool.  If you do not want to install any additional Java tools and you have access to the DC.  You can manually export the certificate from each DC and import tem into the Java keystore  using Method 2.

Method 1: Using InstallCert and Keytool

  1. Download the InstallCert tool created by Andreas Sternbenz from here
  2. If you do not have unzip installed, install it using
    apt-get install unzip
  3. unzip the tool
    unzip InstallCert.zip
  4. Move the contents to your java bin directory
    mv InstallCert* $JAVA_HOME/jre/bin
  5. Acquire the cert from your AD servers like so (note port for LDAPS is usually 636)
    java InstallCert dcServer.domain.com:636
    • If you are prompted for anything, just hit enter to continue
    • If you have more than one server you intended to use for LDAPS authentication, repeat this step
  6. This process will create a file called jssecacerts.  We now want to import this file into our existing cacert file like so
    keytool --importkeystore -srckeystore jssecacerts -destkeystore $JAVA_HOME/jre/lib/security/cacerts -noprompt

    You will be prompted for the password to the cacerts keystore and the jssecacerts keystore.  Enter the password changeit
    Once this is done, you should see a status message saying something around 78 entries where imported.

Method 2: Exporting the Cert from AD and Importing it With Keytool

  1. From a Windows workstation open a run prompt (windows key + r)
  2. Enter mmc and press ok
  3. Go to File -> Add/Remove Snapin..
  4. From the “Available snap-ins” list on the left select “Certificates”
  5. Click the “Add >” button to add Certificates to the Selected snap-ins list on the right
  6. Select “Computer Account” from the options.
  7. Select Local computer if you are on the serve that has the certificate, else select “Another Computer” and enter the host name for the domain controller to get the certificate from
  8. Click finish
  9. Click OK
  10. Expand Certificates folder under Console Root and expand the Personal folder
  11. Click the Certificates folder
  12. Right click the certificate you want to export
  13. Go to All Tasks -> Export
  14. Click Next
  15. Select “Yes, export the private key”
  16. Click Next
  17. Select “Personal Information Exchange – PKCS #12(.PFX)” and check “Export all extended properties” if you can.  If you cannot, select PKCS #7 option above this one, and check the check box below it.
  18. Enter a name for the file
  19. Click ok.  You should now have the server cert
  20. Move the cert to your Linux server using WinSCP or some other file transfer tool
  21. Import the cert by typing
    keytool --import -keystore $JAVA_HOME/jre/lib/security/cacerts -file /path/to/your/cert/file
  22. Enter the password of changeit for the cacerts keystore
  23. If prompted to “Trust this certificate?” type yes and press enter

Setting up the LDAPS Connection

To configure TeamCity to LDAPS for authentication is pretty simple by this point, you simple need to do the following

  1. Edit /opt/TeamCity/.BuildServer/config/main-config.xml and replace the following line
    <login-module class="jetbrains.buildServer.serverSide.impl.auth.DefaultLoginModule" />

    with

    <login-module class="jetbrains.buildServer.serverSide.impl.auth.LDAPLoginModule" />
  2. copy the file ldap-config.properties.dist to ldap-config.properties or just create the file and go to the next step.
    cp ldap-config.properties.dist ldap-config.properties
  3. Edit ldap-config.properties and set the following lines.  If you just created the file, then just set it to the contents below
    # The server(s) to auth against 
    java.naming.provider.url=ldaps://example.com:636/DC=example,DC=com 
    
    # The account to use to search for accounts in LDAP and read their data 
    java.naming.security.principal=CN=username,CN=Users,DC=example,DC=com 
    java.naming.security.credentials=secret_password 
    
    # The base dn to use when searching for users (if your accounts are all over, just leave it blank) 
    teamcity.users.base=CN=users 
    
    # Handles setting the user name.  acceptedLogin has Teamcity use the username in LDAP.  You should use 
    # this if you cannot filter your usernames because of how they are constructed 
    teamcity.users.acceptedLogin= 
    
    # Sets the username in team city to the one in LDAP 
    teamcity.users.username=sAMAccountName 
    
    # Disallow slashes and @s in the username given at login 
    teamcity.auth.loginFilter=[^/\\\\@]+ 
    
    # Synchronize the user against ldap (this runs every hour) 
    teamcity.options.users.synchronize=true 
    
    # Filter what users to match in sync 
    teamcity.users.filter=(objectClass=user) 
    
    # Don't sync groups 
    teamcity.options.groups.synchronize=false 
    
    # Don't create or delete users during synchronization. 
    teamcity.options.createUsers=false 
    teamcity.options.deleteUsers=false 
    
    # The time interval between synchronizations (in milliseconds). By default, it is one hour. 
    teamcity.options.syncTimeout = 3600000 
    
    # The name of LDAP attribute to retrieve user's full name 
    teamcity.users.property.displayName=displayName 
    
    # The name of LDAP attribute to retrieve user's email 
    teamcity.users.property.email=mail 
    
    # Tell team city to only sync users in team city against LDAP (if you have a lot of users in LDAP, this should be turned on). 
    # Only look at 100 users at a time 
    teamcity.users.syncOnlyTeamcityUsers=true 
    teamcity.users.filterPackSize=100
  4. Set www-data as the owner of the file
    chown www-data ldap-config.properties
  5. Start the server
    /etc/init.d/teamcity start
  6. Navigate to the TeamCity site in your browser https://your.server.com:8443
  7. The server will say that the configuration has changed and that it needs a code from the log file.  Get the last ten lines from the server log to get the code.
    tail /opt/TeamCity/logs/teamcity-server.log
  8. The code should be in the output.  Copy this code and past it into the field in your browser and click proceed
  9. You will now be prompted to login.  The account you do this initial login with will be an administrative account.

Configuring Apache to Redirect to Teamcity

This section will cover how to setup Apache to take a url like yourserver.domain.com/teamcity and redirect it to yourServer.domain.com:8443/teamcity.  Users do not want to remember port numbers, so this will be a welcome step for them.  Also if you setup your DNS records correctly, you could even make the teamcity site something like teamcity.domain.com.

This section assumes you have setup Apache to use SSL.

  1. Enable the proxy module
    a2enmod proxy
  2. Enable the rewrite module
    a2enmod rewrite
  3. Edit the load file for the proxy module
    vim /etc/apache2/mods-enabled/proxy.load
  4. Add the following lines
    LoadModule proxy_module /usr/lib/apache2/modules/mod_proxy.so
    LoadModule proxy_http_module /usr/lib/apache2/modules/mod_proxy_http.so
    LoadModule headers_module /usr/lib/apache2/modules/mod_headers.so
  5. Edit the default site to redirect all traffic to SSL
    vim /etc/apache2/sites-available/default
  6. Add the following lines
    RewriteEngine on
    RewriteCond %{SERVER_PORT} ^80$
    RewriteRule ^(.*)$ https://%{SERVER_NAME}$1 [L,R]
    RewriteLogLevel 2
  7. Edit the SSL site file
    vim /etc/apache2/sites-available/default-ssl
  8. Add the following lines
    SSLProxyEngine=on
    ProxyPass /teamcity https://localhost:8443/teamcity
    ProxyPassReverse /teamcity https://localhost:8443/teamcity
  9. Move the web directory of TeamCity to a directory that will work for this
    mv /opt/TeamCity/webapps/ROOT /opt/TeamCity/webapps/teamcity
  10. Restart Apache
    /etc/init.d/apache2 restart
  11. Open a web browser and navigate to https://yourServer.domain.com/teamcity
  12. At this point you should be looking at the login window for TeamCity

Installing OCI8 on Ubuntu

If you are planning on using the oracle functions in PHP, you will need to install OCI8. Before you begin you will need to create an account with oracle or have access to the instant client basic install file and sdk file for your linux architecture.

Prerequisites for installing OCI8

  • sudo (you will be doing the command line parts as sudo).
  • An account on the oracle  website so you can download the instant client files.
  • A working Apache and PHP installation.

Installing the Oracle Instant Client Files

  1. Download the Oracle Instant Client for Linux (Both the Basic and SDK versions) from here.  Make sure you choose the correct architecture for your Ubuntu installation (32bit is x86, 64bit is x86_64).
  2. Copy the files just downloaded to your Ubuntu server using scp or the WinSCP application for windows.
  3. ssh into your ubuntu server using your favorite ssh client
  4. Install PECL, PHP Development files, Build Essential, Unzip, and the AIO Library
    apt-get install php-pear php5-dev build-essential unzip libaio1
  5. Create the oracle directory
    mkdir /opt/oracle
  6. Move the downloaded files
    mv instantclient-* /opt/oracle
  7. Unzip the files using the unzip command (unzip <filename>)
  8. Rename the created directory
    mv instantclient_11_2 instantclient
  9. cd into the instant client directory and create the following soft links
    ln -s libclntsh.so.11.2 libclntsh.so
    ln -s libocci.so.11.2 libocci.so
  10. cd to  /opt
  11. set the permissions on the directory
    chown -R root:www-data /opt/oracle
  12. Add the instant client to the the ld config files
    echo /opt/oracle/instantclient > /etc/ld.so.conf.d/oracle-instantclient
  13. Update the Dynamic Linker Run-Time Bindings
    ldconfig

Installing OCI 8 and Configuring Apache

  1. Install OCI8 using PECL
    pecl install 0ci8
  2. When you are pompted for the client location enter the following:
    instantclient,/opt/oracle/instantclient
  3. Add the extension to the php.ini files
    echo extension=oci8.so >> /etc/php5/apache2/php.ini
    echo extension=oci8.so >> /etc/php5/cli/php.ini
  4. Restart Apache
    /etc/init.d/apache2 restart
  5. Verify the installation of the OCI8 library by creating a simple phpinfo page.
    touch info.php; echo "<?php phpinfo(); ?>" > info.php
  6. Navigate to the page using your favorite web browser and confirm the OCI8 section is present.

Although the libraries to work with Microsoft SQL Server used to be included with PHP, this is no longer the case. If you are using an IIS server, Microsoft has been gracious enough to provide drivers that use the native SQL client called SqlSrv; however, Linux users do not have access to this driver. To work with SQL Server 2008 on linux, you will need to install the FreeTDS libraries.

Installation and Configuration of FreeTDS

Note: You will need sudo access to do this procedure

  1. Install the Sysbase library for PHP (FreeTDS is provided with this)
    sudo apt-get install php5-sybase
  2.  Configure the freetds.conf file to use version 8.0 (this is necessary to work with MSSQL 2008, if you do not do this you will encounter odd behavior)
    sudo vim /etc/freetds/freetds.conf
  3. Change this line under the [global] section
    ;       tds version = 4.2

    to

    tds version = 8.0
  4. Restart Apahce
    sudo /etc/init.d/apache2 restart

Redmine is a robust project management solution for software development teams to use.  Some of the features that Redmine offers are SVN integration, Issue tracking, RSS feeds, and LDAP Authentication.  Below I will detail the installation process, and pre-installation requirements for setting up and configuring your own Redmine server.

Before we begin though you should be comfortable with the following:

  • linux command line
  • using sudo
  • using vim or vi (we are not doing much, just editing and saving)

For this tutorial, all commands are done using superuser (sudo su), or you can prefix sudo to each command.

Package Installations

Optional Installs

Since this tutorial is for a completely new install of Ubuntu with no additional packages selected during installation, there are a few packages that need to be installed.

Install Vim

Throughout this tutorial, I will be using vim to edit files.  If you are unfamiliar with vim or vi you may want to use another file editor.

  1. Install Vim
    apt-get install vim

Install SSH

If SSH has not been installed on the Ubuntu server you are using, it is recommend that you install it.  Although not necessary, it makes the rest of the tutorial easier.

  1. Install SSH
    apt-get install ssh
  2. Configure SSH to deny root logon.
    vim /etc/ssh/sshd_config

    Change the line

    PermitRootLogin yes

    to

    PermitRootLogin no
  3. Restart SSH
    /etc/init.d/sshd restart

Now you will be able to connect to your Ubuntu server using an ssh client like putty.

Required Installs

Since Redmine will be working with subversion, Apache, PostgreSQL (you can use MySQL if you like), ruby, rails, passenger, etc; there are a lot of packages to install.

Apt-get Installs

  1. Install the following packages using apt-get install
    • apache2-threaded-dev
    • build-essential
    • libapache2-mod-passenger
    • libapache2-mod-perl2
    • libapache2-svn
    • libapache-dbi-perl
    • libauthen-simple-ldap-perl
    • libcurl4-openssl-dev
    • libdbd-pg-perl
    • libdbd-pg-ruby1.8
    • libdigest-sha1-perl
    • libgemplugin-ruby1.8
    • libmagick9-dev
    • libruby1.8-extras
    • php5
    • php5-curl
    • php5-dev
    • php5-pgsql
    • postgresql
    • rails
    • rake
    • ruby1.8-dev
    • rubygems1.8
    • sendmail
    • subversion

Gem Installs

Redmine runs with ruby on rails and uses several ruby gems.  To install these gems, run the following commands.

  1. gem install rails -v=2.3.5
  2. gem install rack -v=1.0.1 (should been installed by the previous command)
  3. gem install rmagick
  4. gem install passenger
  5. gem install pg
  6. gem install -v=0.4.2 i18n

Redmine Install

This section details the installation and configuration setups for the command line part the installation.

PostgreSQL database setup

  1. Sudo as the user postgres and connect to the default database
    sudo –u postgres psql postgres
  2. Create the redmine database user “redmine”
    CREATE ROLE redmine LOGIN ENCRYPTED PASSWORD 'somePassword' NOINHERIT VALID UNTIL 'infinity';
  3. Create the database for Redmine called redmine
    CREATE DATABASE redmine WITH ENCODING='UTF8' OWNER=redmine;
  4. Exit out of psql
    \q

Installing Redmine Files

  1. Create the Redmine install directory
    mkdir /opt/redmine
  2. Checkout the stable Redmine installation
    svn co http://redmine.rubyforge.org/svn/branches/1.1-stable /opt/redmine
  3. Enter the Redmine installation directory
    cd /opt/redmine
  4. Setup permissions for Apache
    chown -R www-data:www-data files log tmp public/plugin_assets/
    chmod -R 755 files log tmp public/plugin_assets/

Configure the Redmine Database Settings

  1. Change directory to the config directory
    cd /opt/redmine/config
  2. Create the database configuration file
    cp database.yml.example database.yml
  3. Edit the database configuration file
    vim database.yml
  4. Make the production setting look like this
    production:
      adapter: postgresql
      database: redmine
      host: localhost
      username: redmine
      password: your_redmine_database_password
      encoding: utf8
  5. Save the file (press esc, type :wq, press enter)

Configure the Redmine Email Settings

  1. Change directory to the config directory
    cd /opt/redmine/config
  2. Create the email configuration file
    cp email.yml.example email.yml
  3. Edit the email configuration file
    vim email.yml
  4. Make the production setting look like this
    production:
      delivery_method::sendmail
      #smtp_settings:
      #address: smtp.example.net
      #port: 25
      #domain: example.net
      #authentication::login
      #user_name: "redmine@example.net"
      #password: "redmine"
  5. Save the file

Setup the Redmine Database

  1. Change directory to /opt/redmine
    cd /opt/redmine
  2. Generate the session store
    rake generate_session_store
  3. Migrate the database (this is a command to run)
    RAILS_ENV=production rake db:migrate
  4. Load the default information into the database
    RAILS_ENV=production rake redmine:load_default_data
  5. You will be prompted for what language you would like to use.  Press enter for English.
  6. Press Enter again

Apache Configuration

Setup passenger

  1. Enable Passenger
    a2enmod passenger
  2. Edit the Passenger configuration file
    vim /etc/apache2/mods-available/passenger.conf
  3. Add the line
    PassengerDefaultUser www-data

Link the Redmine.pm file

  1. Change directory to Apaches perl module directory
    cd /usr/lib/perl5/Apache
  2. Create a softlink to the Redmine.pm file
    ln -s /opt/redmine/extra/svn/Redmine.pm Redmine.pm

Link to the Redmine public folder

  1. Change directory to the document root for Apache
    cd /var/www
  2. Create a softlink to the public folder of redmine
    ln -s /opt/redmine/public redmine

Create the SVN Repositories Directory and Set Permissions

  1. Create the repository directory
    mkdir -p /var/svn/repos
  2. Set permissions for Apache
    chown -R www-data:root /var/svn

Create a SSL Certificate for the Site

If your site already has a signed SSL certificate from a legitimate certificate authority, you can skip this part of the tutorial.

  1. Create a self signed ssl cert
    openssl req -new -x509 -days 365 -nodes -out /etc/ssl/certs/cert.pem -keyout /etc/ssl/certs/cert.pem
  2. Enter in your information companies’ for each prompt, or accept the defaults

Enable Needed Apache Modules and Create the Redmine Site

  1. Enable the following Apache modules using the a2enmod command
    • dav_svn
    • perl
    • ssl
    • rewrite
  2. Create the Redmine site file
  3. vim /etc/apache2/sites-available/redmine
  4. Set the contents of the redmine site file to
  5. <VirtualHost _default_:443>
         ServerAdmin webmaster@localhost
         DocumentRoot /var/www
         PerlLoadModule Apache::Redmine
         PerlLoadModule Authen::Simple::LDAP
         # PerlLoadModule IO::Socket::SSL
         RailsEnv production
         RailsBaseURI /redmine
        <Directory /opt/redmine/public>
             Options FollowSymLinks
             AllowOverride none
             Order deny,allow
             Allow from all
        </Directory>
        #This holds the configuration for the web accessible svn     
        <Location /svn>
            DAV svn
            SVNParentPath "/var/svn/repos"
            AuthType Basic
            AuthName redmine
            Require valid-user
            PerlAccessHandler Apache::Authn::Redmine::access_handler
            PerlAuthenHandler Apache::Authn::Redmine::authen_handler
            RedmineDSN "DBI:Pg:dbname=redmine;host=localhost"
            RedmineDbUser "redmine"
            RedmineDbPass "your_redmine_password"
            RedmineCacheCredsMax 50
        </Location>
        #Used by reposman.rb to create repos for new Redmine projects 
        <Location /sys>
            Order deny,allow
            Allow from 10.30.100.202, 127.0.0.1
            Deny from all
        </Location>
        SSLEngine on
        SSLCertificateFile /etc/ssl/certs/cert.pem
        SSLCertificateKeyFile /etc/ssl/certs/cert.pem
    </VirtualHost>
  6. Save the file
  7. Restart Apache (if all is good, Apache should start without error)
    /etc/init.d/apache2 restart

Setup Automatic Subversion Repository Creation

  1. Make the Redmine log file location
    mkdir /var/log/redmine
  2. Create the log file
    touch /var/log/redmine/reposman_errors.log
  3. Edit the Crontab (you may want to make a backup first)
    crontab -e
  4. Select an editor from the presented list (I selected nano, number 2)
  5. Add this line to the bottom of the file (this is one continuous line broken up for readability),  This line tells the cron to run this job every minute.
    * * * * * ruby /opt/redmine/extra/svn/reposman.rb -r https://your_server_ip/redmine -s
    /var/svn/repos -o www-data --url file:///var/svn/repos >>
    /var/log/redmine/reposman_errors.log #Add new repos for projects
  6. Press Ctrl + O to save the file
  7. Press Enter to confirm the file name
  8. Press Ctrl + X to exit nano

Optional Configurations

Disable Directory Browsing in Apache

We do not want people to be able to browse the files in our webroot, so let’s block that.

  1. Edit the default site configuration file
  2. vim /etc/apache2/sites-available/default
  3. Under the <Directory /var/www> section add a – infront of the word Indexes so the line looks like this
    Options -Indexes FollowSymLinks MultiViews
  4. Do the same for /etc/apache2/sites-available/default-ssl
  5. Restart Apache
    /etc/init.d/apache2 restart

Redirect all HTTP traffic to HTTPS

  1. Edit the default site configuration file
    vim /etc/apache2/sites-available/default
  2. Add the following lines after the DocumentRoot statement (do not change SERVER_NAME or SERVER_PORT, they are variables)
    RewriteEngine on
    RewriteCond   %{SERVER_PORT} ^80$
    RewriteRule   ^(.*)$ https://%{SERVER_NAME}$1 [L,R]
    RewriteLogLevel 2
  3. Restart Apache
    /etc/init.d/apache2 restart

Configuring The Rest of Redmine Via a Web Browser

  1. Open Firefox or you favorite browser and navigate to https://yourIPAddress/redmine.  For example, https://example.page.com/redmineYou should see this page:
    "The Redmine Uconfigured Main Page"

Change the admin account information (defaults are bad!)

  1. Click the “sign in” link in the upper right corner of the page.
  2. Sign in with the following credentials
    username: admin
    password: admin
  3. Click the “Administration” link in the upper left part of the page
  4. Click the “Users” link
  5. Click the username (which is a link) of the admin user
  6. Change the accounts information (especially the password)
  7. Click the save button

Configure the Redmine Server Settings

  1. After signing in as a user with administrator rights, click the “Administration” link in the upper left part of the screen
  2. Click the Settings link
  3. Under the General tab
    1. Change Host Name and Path from localhost:3000 to your.server.com/redmine (ex. example.demo.com/redmine)
    2. Change Protocol from HTTP to HTTPS
    3.  Click the Save button
  4. Under the Authentication tab
    1. Check “Authentication Required”
    2. Check “Enable REST Web Service”
    3. Change Minimum Password Length from 4 to 8
    4. Click the Save button
  5. Under the Projects tab
    1. Change the “Role given to non-admin users who creates a project” from “–Please Select–” to Manager
    2. Click the Save button
  6. Under the Email Notifications tab
    1. Change the “http://hostname/my/account&#8221; part in the “Emails footer” field to https://yourSeverAddress/redmine/my/account (for example: https://demo.example.com/redmine/my/account)
    2. Click the Save button
  7. Under the Repositories tab
    1. Check “Enable WS for repository management”
    2. Check “Filesystem”
    3. Click the Save button

Add LDAP Authentication (Optional)

  1. Click the Administration link in the upper left corner of the page
  2. Click the LDAP Authentication link
  3. Click the New Authentication Mode link
  4. Fill out the form as shown below, substituting in your own Active Directory domain information.  Do not alter the values for the Attributes section, these are domain type specific.
  5. If you want to manually create users who authenticate against AD, then uncheck “On-the-fly  user Creation”, else their account will be created on the first login attempt.
  6. Click the Create button.  If all went well, you should be able to authenticate against Active Directory.
  7. When creating new accounts, you will now have to select what authentication source you want to use.

Creating Your First Redmine Project

  1. Click the Projects link in the upper left corner of the page
  2. Click the New Project link
  3. Enter a Name for the project
  4. Enter a Identifier for the project
  5. The identifier is user to access your svn repository, so if you have an identifier of tstprj your
  6. repository will be available at https://yourServer/svn/tstprj
  7. If you want a private project, only accessible by the members of the project, uncheck Public
  8. Click the Save button
  9. Click the Projects link in the upper left corner of the page
  10. Click the link to the project you just made
  11. Click the Settings link
  12. Click the Members tab
  13. Select the members you would like to add and their role for the project form the list on the left
  14. Click the Add button
  15. Ignore repository tab as the cronjob we created earlier should create an svn reposity and link it to the project for you.

If you have made it this far and everything is working as expected, then Congratulations!!! You have setup your very own Redmine server with Subversion integration.  If things are not working, I recommend looking at the HowTo’s other documentation at redmine.org

Zend Framework offers a very robust solution for almost any kind of web site; however, there are times when all you want to do is connect to a database without the need for the rest of the framework.  Bellow I will show a quick example of how to connect to two different databases using the Zend_Db, Zend_Config_Ini, and Zend_Loader  classes.

First you need to define your ini file (make sure to store this file out side of the web root, mine is stored at /var/)

[production]
; PostgreSQL database connection info
db.pgsqlServer.adapter = PDO_PGSQL
db.pgsqlServer.config.host = server_ip_or_url
db.pgsqlServer.config.dbname = database_name
db.pgsqlServer.config.username = "dbuser"
db.pgsqlServer.config.password = "dbpassword"
db.pgsqlServer.config.driver_options.PDO::ATTR_ERRMODE=PDO::ERRMODE_EXCEPTION

; Microsoft SQL Server database connection info
db.mssqlServer.adapter = PDO_MSSQL
db.mssqlServer.config.pdoType = dblib
db.mssqlServer.config.host = server_ip_or_url
db.mssqlServer.config.dbname = database_name
db.mssqlServer.config.username = "mssql_user"
db.mssqlServer.config.password = "mssql_password"
db.mssqlServer.config.driver_options.PDO::ATTR_ERRMODE=PDO::ERRMODE_EXCEPTION

From the above ini file contents, there are a few things to take note of:

  1. You need to define the pdoType as dblib when connecting to MSSQL with the  PDO Adapter (the default type mssql is not available in the php5_sybase package)
  2. You can set driver_options (attributes) for the PDO connection, some of the attributes you can set are
    1. PDO::ATTR_ERRMODE = PDO::ERRMODE_EXCEPTION             (throw exceptions)
    2. PDO::ATTR_PERSISTANT = true                                                  (make persistent connections)
    3. PDO::ATTR_DEFAULT_FETCH_MODE=PDO::FETCH_ASSOC   (return an associative array when fetching query result by default)

Next, create your php file that will be using the database connections. For this example, my php file is located at /var/www/example_project and the Zend libraries are located at include/libs/Zend

<?php
//set the include path for Zend Framework (the include path is not defined in php.ini)
set_include_path('include/libs' . PATH_SEPARATOR . get_include_path());

//Include the Zend_Loader class
require_once 'Zend/Loader.php';

//load the Zend classes for use
Zend_Loader::loadClass('Zend_Config_Ini');
Zend_Loader::loadClass('Zend_Db');

//load the configuration file
$config = new Zend_Config_Ini('/var/my_example.ini', 'production');

//try to connect to and work with the postgres database
try
{
     //connect to postgresql
     $dbh = Zend_Db::factory(
         $config->db->pgsqlServer->adapter,
         $config->db->pgsqlServer->config->toArray()
     );

     //set the instructor id number to use for the query
     $instructorID = 1234;

     //define the query
     $sql = "SELECT Full_Name From Trained WHERE train_id = :idNumber";

     //prepare the query
     $stmt = $dbh->prepare($sql);

     //bind the instructor id as an integer data type to the query
     $stmt->bindParam(':idNumber', $instructorID, PDO::PARAM_INT);

     //execute the query
     $stmt->execute();

     //if an instructor with the given id exits
     if ($stmt->rowCount() > 0) {

         $row = $stmt->fetch(PDO::FETCH_ASSOC);
         echo "The instructors full name is: ",$row['full_name'];
     }

     //clear the statement
     $stmt = null;

     //close the database connection
     $dbh->closeConnection();

}
catch (PDOException $e)
{
     //echo $e->getMessage();
     echo "There was an error querying the database";
}
catch (Zend_Db_Adapter_Exception $e)
{
     //echo $e->getMessage();
     echo "Unable to connect to the database";
}
catch (Exception $e)
{
     echo $e->getMessage();
}
?>

In the above example, I only connected to and used the Postgres database since the code for using either database is almost exactly the same.  The only thing you would need to change in the above code to use the MSSQL database would be to change the pgsqlServer part in the Zend_Db::factory() parameters to mssqlServer.

In closeing the catch blocks at the bottom the php file do the following

  • PDOException – Catches any exception thrown by PDO while working with the database (in this example, this pertains to querying the database)
  • Zend_Db_Adapter_Exception – Catches any exception thrown while trying to connect to the database (or while using Zend_Db_Adapter, the $dbh variable)
  • Exception – Catches any other exceptions I missed
Follow

Get every new post delivered to your Inbox.