Tools

CNTLM

CNTLM is an authentication proxy that can solve problems with applications trying to get through a Windows proxy that requires authentication. I need this to clone from GitHub repos when behind a workplace proxy server. Getting it installed, however, may involve a little fancy footwork.

  1. Rename the .exe to something else as some policies won’t allow it to be installed. How about mostly_harmless.exe?
  2. Use the command line options -H -u -d to get the hashes for your config file: mostly_harmless.ini
  3. Launch the application: mostly_harmless -c mostly_harmless.ini
  4. Edit your git.config: [http] proxy=http://127.0.0.1:3128

Hurrah!

NuGet and chocolatey behind a proxy

NuGet historically didn’t handle company proxies, a common enough issue, which has led to a lot of guidance on the ‘net that is now out of date on how to deal with a proxy that isn’t playing nicely with your default credentials. Here are two tips: the first will let you get chocolatey installed and the second will let you configure NuGet so that chocolatey can do its goodness.

install chocolatey with windows auth proxy

The above gist will let you install chocolatey from a PowerShell command line. I used the normal download URL like this:

Set-ExecutionPolicy -ExecutionPolicy Unrestricted;$a=new-object net.webclient;$a.proxy.credentials=[system.net.credentialcache]::defaultnetworkcredentials;$a.downloadstring('https://chocolatey.org/install.ps1')|iex

NuGet configuration settings

From the linked configuration settings above I added:

.\nuget config -Set http_proxy=company-proxy:8080
.\nuget config -Set http_proxy.user=mylogin
.\nuget config -Set http_proxy.password=secret

Note that the password is encrypted in your NuGet.config so you can’t edit it directly.

Now I can use chocolatey without seeing:

Please provide proxy credentials:
UserName: Cannot prompt for input in non-interactive mode.

I believe that PowerShell and NuGet are trying to present the default web credentials rather than the default network credentials and in our case the proxy we lurk behind isn’t satisfied with that, thus the need to be explicit. It’s possible that NuGet was failing to present any credentials; I could investigate with Fiddler but these fixes have solved it for me. YMMV.

Mercurial with a Subversion central repository

This How-To is for developers wishing to use Mercurial (Hg) on their Windows development boxes when the source code is in a central Subversion (SVN) repository.

Install TortoiseHg.

In a Windows development environment the easiest way to get started with Hg is to install TortoiseHg. The Windows Explorer extension not only makes it easy to work with files by using icon overlays and context menus but also packages up Python, the SVN bindings and a large number of Hg extensions that will prove very useful.

If you are behind a proxy you must first configure Hg with the proxy settings. Use the TortoiseHg context menu and Global Settings > Proxy.

Working with SVN.

Given that you are working against a central SVN repository then Hg has a number of extensions that can talk to the SVN repo. I use HgSubversion. To install this extension you need to clone the hgsubversion repo:

hg clone http://bitbucket.org/durin42/hgsubversion c:\hgsvn

Use the TortoiseHg context menu and Global Settings > Edit File.

[extensions]
hgsubversion = c:\hgsvn

Start with a single project. It’s better to clone the whole repo, the trunk, branches and tags, rather than just a branch. This way you only need to do the clone once – clones can take a while. Create a project folder, i.e. c:\development\local\myproject, change to it and then clone the SVN repo:

hg clone sv://subversion/myproject/

Ignore.

It’s best to set up the ignore list for Hg now. Save this as .hgignore and copy it to the root of your local Hg clone (next to .hg).

# Mercurial .hgignore file template by Abdullin.com
# For syntax see: http://linux.die.net/man/5/hgignore
# Source: http://bitbucket.org/abdullin/snippets/

# Visual Studio
glob:*.user
glob:*.suo
glob:*.cache
glob:_ReSharper.*/
relre:/obj/
relre:/bin/
relre:/Bin/

# Subversion
glob:.svn/
glob:_svn/

# Build structure
relre:^Build/

# Misc
glob:Thumbs.db
glob:*.bak
glob:*.log

Local Development.

If you use local branches and merge them in your local Hg repo you won’t be able to push changes to SVN.

If you commit often then you will end up with lots of changesets to push to SVN that will make it harder to see what the intention of your update was.

A good way to solve both problems is to submit patches to the central repo. Hg has an extension for patch management called Mercurial Queues (MQ).

Mercurial Queues.

Mercurial Queues (MQ) provides patch management integrated with Hg. This is incredibly useful for packaging up a lot of small changes that you record as you work locally into a single submission to the central repo. You can also work on a number of patches concurrently.

Enable the MQ extension in Hg, TortoiseHg > Global Settings > Extensions and check ‘mq’. In the mercurial.ini set diffs to use the git format.

[diff]
git = True

In the Hg Repository Explorer you can create a new patch from the latest revision. This means you will need to commit something you want to go into a patch before you can create the patch. The other way is to use the command line.

hg qnew mylatestpatch

qnew will include the latest changes found in the working directory.

qfold

One way to work with plenty of local commits but to have a single, tidy submission to the central repo is to use MQ’s qfold to create a patch.

Once you have some changesets you wish to publish you convert them into patches. In Hg Repository Explorer use the context menu on the latest changeset first: Mercurial Queues > Import Revision to MQ. Repeat for each of the changesets. Next, unapply the latest patches until just the earliest is still applied. Now at the command line use qfold. When you refresh Hg Repository Explorer you will see a single patch. Use the patch context menu’s ‘Finish Applied’ to change the patch into a single changeset you can push to the central repo.

For example, you have three changesets: 16; 17 and 18. You then create three patches: 16.diff; 17.diff and 18.diff. You unapply 17.diff and 18.diff then go to the command line:

hg qfold 17
hg qfold 18

Now you have a single patch, 16.diff, that contains all the work commited in the original three changesets.

SpecFlow

Acceptance Tests for User Stories allow the Product Owner to more easily say whether or not they accept a Story as ‘Done’. Also, Acceptance Tests can be used in Behaviour Driven Development (BDD) to provide an “outside in” development process that complements the “inside out” coding style of Test Driven Development (TDD).

SpecFlow brings Cucumber BDD to .NET without the need for a Ruby intermediary like IronRuby.

In your Tests project add a Features folder. SpecFlow installs some templates into VS.NET so add a new SpecFlowFeature and fill it out like the following example, InterestRatings.feature :

Feature: Interest Ratings
	In order to manage the Interest Ratings
	As a Trader
	I want an Interest Ratings screen

@view
Scenario: View the Interest Ratings
	Given a repository of Interest Rating records
	When I view the Interest Rating screen 
	Then the full list is created

@create
Scenario: Create an Interest Ratings record
	Given a repository of Interest Rating records
	When I create an Interest Rating with name Test 
	And Interest Rating code 4
	Then the Interest Rating is saved to the repository with 
                 name Test and code 4

The scenarios are the tests. The format is a clear Given-When-Then description. As you create the scenario the .feature.cs will be updated for you.

Now you need to link up the statements in your scenario to steps that the unit test framework can execute. Create a Steps folder under Features and add a SpecFlowStepDefinition. You’ll find the generated file has some useful placeholders to get you started. Here, for example, is InterestSteps.cs :

    [Binding]
    public class InterestSteps
    {
        private IInterestRatingService interestService;
        private InterestRatingViewModel interestRatingViewModel;
        private InterestRating rating;
        private Mock<IValidationService> validationService 
                      = new Mock<IValidationService>();
        private Mock<ILoadScreen> loadScreen 
                      = new Mock<ILoadScreen>();
        private Mock<IServiceLocator> serviceLocator 
                      = new Mock<IServiceLocator>();
        private Mock<IRepository<InterestRating>> interestRepository 
                      = new Mock<IRepository<InterestRating>>();
        private List<InterestRating> interestRatings;

        [Given("a repository of Interest Rating records")]
        public void GivenARepositoryOfInterestRatingRecords()
        {
            Mock<IValidator> validator = new Mock<IValidator>();
            this.serviceLocator
                     .Setup(s => s.GetInstance<IValidationService>())
                     .Returns(this.validationService.Object);
            this.validationService
                     .Setup(v => v.GetValidator(
                         It.IsAny<InterestRating>()))
                     .Returns(validator.Object);
            this.serviceLocator
                     .Setup(s => s.GetInstance<IEventAggregator>())
                     .Returns(new EventAggregator());
            this.serviceLocator
                     .Setup(s => s.GetInstance<ILoadScreen>())
                     .Returns(this.loadScreen.Object);
            ServiceLocator.SetLocatorProvider(() => this.serviceLocator.Object);
            this.interestRatings = 
                      InterestRatingMother
                          .CreateGoodInterestRatingMother()
                          .InterestRatings
                          .Cast<InterestRating>().ToList();
            this.interestRepository
                     .Setup(s => s.GetAll())
                     .Returns(this.interestRatings);
            this.interestService 
                  =  new InterestRatingService(
                                this.interestRepository.Object);
        }


        [When("I view the Interest Rating screen")]
        public void WhenIViewTheInterestRatingScreen()
        {
            this.interestRatingViewModel 
                  = new InterestRatingViewModel(this.interestService);
            this.interestRatingViewModel.Load();
        }

        [When("I create an Interest Rating with name (.*)")]
        public void WhenICreateAnInterestRatingWithName(string name)
        {
            this.interestRatingViewModel 
                  = new InterestRatingViewModel(this.interestService);
            this.interestRatingViewModel.Load();
            this.interestRatingViewModel.Add();
            this.rating 
                  = this.interestRatingViewModel
                              .InterestRatings[
                                   this.interestRatingViewModel
                                        .InterestRatings.Count - 1];
            this.rating.InterestRatingName = name;
        }

        [When("Interest Rating code (.*)")]
        public void AndInterestRatingCode(int code)
        {
            this.rating.InterestRatingCode = code;
            this.interestRatingViewModel.Save();
        }

        [Then("the full list is created")]
        public void ThenTheFullListIsCreated()
        {
            Assert.That(
                this.interestRatings.Count 
                      == this.interestRatingViewModel
                               .InterestRatings.Count);  
        }


        [Then("the Interest Rating is saved to the repository 
         with name (.*) and code (.*)")]
        public void ThenTheInterestRatingIsSavedToTheRepository(
                           string name, int code)
        {
            InterestRating rating 
                = (from m in this.interestRatingViewModel.InterestRatings
                   where m.InterestRatingName.Equals(name)
                   select m).Single();

            Assert.That(
                rating.InterestRatingName.Equals(name),
                "The interest rating name was not saved.");

            Assert.That(
                rating.InterestRatingCode == code,
                "The interest rating code was not saved.");
        }
    }

In particular, notice the reuse of steps, for example GivenARepositoryOfInterestRatingRecords(), and the use of variable placeholders like (.*) to allow the passing of variables into the tests.

BDD wraps TDD. A reasonable flow would be to start with the Story, write up the Acceptance Tests and sketch out some of the steps. As you sketch out the steps you can see what unit tests you need so you go and develop the code using TDD. Once your code is ready and all the unit tests are passing you can integrate the layers with the BDD tests and when those are passing you have fulfilled your Acceptance Test.

Gherkin parsers for SpecFlow are on the way as are VS.NET language plugins (Cuke4VS – currently this crashes my VS.NET 2008).

Cuke4Nuke is another Cucumber port that is worth looking at.

The readability of the Features makes it easy to take Acceptance Tests from User Stories so that the Product Owner and Stakeholders can see what the system is doing. The “outside in” nature of the creating the code gives focus to fulfilling the User Story.

PartCover

Setting up PartCover for .NET unit test code coverage takes a little work.

Trying to running PartCover.exe on x64

“Retrieving the COM class factory for component with CLSID {FB20430E-CDC9-45D7-8453-272268002E08} failed due to the following error: 80040153.”

This is because the COM class requested is 32-bit only and PartCover is running as a 64-bit app.

Setting environment for using Microsoft Visual Studio 2008 x86 tools.

C:\> "%VS90COMNTOOLS%\vsvars32"

Microsoft (R) .NET Framework CorFlags Conversion Tool.

C:\> CORFLAGS /32BIT+ /FORCE path\to\PartCover\PartCover.exe

Microsoft (R) .NET Framework Strong Name Utility

C:\> sn -k PartCover.Console.snk

-k [<keysize>] <outfile>
Generate a new key pair of the specified size and write it into <outfile>.

C:\> sn -R PartCover.exe PartCover.Console.snk

-R[a] <assembly> <infile>
Re-sign signed or partially signed assembly with the key pair in <infile>. If -Ra is used, hashes are recomputed for all files in the assembly.

Running PartCover

This is an example command line to create an Xml report of code coverage.

.\tools\PartCover\PartCover.exe
--target .\tools\NUnit\nunit-console-x86.exe
--target-work-dir .\src\Tests\
--target-args Alpha.Modules.Sam.Tests\bin\Alpha.Modules.Sam.Tests.dll
--output .\docs\PartCoverReport.xml
--include [Alpha*]*
--exclude [*Tests]*

Viewing the report

A new viewer for the reports is required (the stylesheets aren’t very good) – ReportGenerator.

C:\>ReportGenerator.exe
path\to\PartCoverReport.xml
path\to\output_dir

PartCover.cmd

A command script with argument placeholders.

.\tools\PartCover\PartCover.exe
--target .\tools\NUnit\nunit-console-x86.exe
--target-work-dir %1
--target-args %2
--output .\docs\PartCoverReport.xml
--include %3
--exclude [*Tests]*

.\tools\ReportGenerator\ReportGenerator.exe
.\docs\PartCoverReport.xml
.\docs\PartCover\

Using PartCover from VS.NET

VS.NET > Tools > ExternalTools…

Title: PartCover
Command: $(SolutionDir)..\PartCover.cmd
Arguments: $(BinDir) $(BinDir)$(TargetName)$(TargetExt) [*Module*]*
Initial directory: $(SolutionDir)..
Use Output window [check]
Prompt for arguments [check]

Now there is a “PartCover” option in the Tools menu. Select a Test project and select Tools > PartCover. In the displyed command arguments change “Module” to the name of the project, i.e. “Sam”, and run.

The report is written out to: .\docs\PartCover\index.htm

(I would launch IE automatically but haven’t added that to the command script yet.)

x86/x64 shuffle

64 bit versus 32 bit is stinging us all over the place. Now we have some x64 dev machines (nice) and others using x86 machines. The solution is to set the build platform target of each and every project file to x86.

Also, the build server (TeamCity) is on an x64 box. DLLs that were being called had to be updated like NUnit and the Oracle data client. Then NBehave had to be updated as it’s version of NUnit wasn’t exactly the same so we have a nightly build of NBehave!

There are even problems with some of our tools like this problem with PartCover.

Installing Aptana Studio 1.2 on Ubuntu 8.10 x86

I’m using Aptana Studio for some PHP/RoR/Python web development and I have Win XP x64 on a Rock xtreme64 D900K but my Sony Vaio TX3 uses Ubuntu (Compiz Fusion is irresistible). To keep the development environments the same I’ve created an Ubuntu guest virtual machine (VM) for PHP work on the Win XP host machine. Aptana is easy to install under Windows but it’s less easy under Linux so it’s worth documenting how this can be done with the current releases.

VirtualBox VM with Windows XP x64 Host and Ubuntu 8.10 Desktop x86 Guest (procedure should work for any Ubuntu 8.10 install).

  1. download and unarchive Aptana Studio Standalone for Linux (v.1.2)
  2. place the aptana folder under /usr/local/aptana

    sudo mv ./aptana/ /usr/local/

  3. install Java v.1.6

    sudo apt-get install sun-java6-jre sun-java6-plugin sun-java6-fonts

  4. install xulrunner 1.8.* with the Synaptic Package Manager
  5. save the following script as /usr/local/aptana/runAptana.sh

    #!/bin/bash
    export MOZILLA_FIVE_HOME=/usr/lib/xulrunner
    /usr/local/aptana/AptanaStudio

  6. Make your script executable

    chmod +x runAptana.sh

  7. download and unarchive the Aptana icons: http://support.aptana.com/asap/secure/attachment/10398/aptana_icons.zip
  8. move the aptana_icons folder to under the aptana folder
  9. create a desktop and/or panel launcher

    name: Aptana Studio
    command: browse to /usr/local/aptana/runAptana.sh
    icon: use aptana48x48.png for the desktop launcher
    comment: web development

  10. click on launcher 🙂

Note: I didn’t need to install Firefox v.2. The current version of FF is 3.0.5.

Technorati Tags: ,

USB drive portable apps

My 1GB USB thumb drive (or flash drive) is over two years old and it’s seen a lot of action. In fact, too much, as it’s cooked. I mean this quite literally – it runs very hot and misreports itself, completely dropping off the other day. It’s being a bit quirky now but I have a replacement, a SanDisk Cruzer Micro Skin 8GB. This seems like a good time to record how it’s set up as it’s had a lot of tweaks over the last couple of years. I ran with a 256MB thumb drive before that so there’s been quite a bit of expansion in how I use my usb stick. The main thing that’s changed is that it now has exclusively free software on it.

I’ve created three top-level folders: Data, Downloads and Program Files. The only files at the root are a ReturnIfLost.txt, a launch.bat and an AutoRun.inf. As the drive sits on my keyring the ReturnIfLost file has anonymous details in it (and the offer of a small reward)!

xxx@yahoo.co.uk
+44xxxxxxxxxx
20UKP reward for safe return!

The launch.bat is:

@echo off
REM Launch portable apps
cd "Program Files\PStart"
start PStart.exe

The AutoRun.inf is:

[AutoRun]
open=Launch.bat
action=Launch

Obviously, some app called PStart is required. This an extremely useful utility from Pegtop Software that puts a panel into your system tray from which you can launch any of the programs you’ve set up on your thumb drive. Within this panel I’ve created four top-level folders: Development; Internet; OpenOffice and Utilities (which could do with some sub-folders as it runs to over thirty apps now). Where these have ‘Portable’ in their names I’ve removed it as that gets tired, quick.

Development

CassiniWebServer
CSVed
GIMP
Notepad++
NVU
QueryExpress
SQLiteSpy
UPX executable packer

Internet

AM-DeadLink
FileZilla
Firefox
GoogleEarth
Pidgin
ProxyGet
Skype
Tor
utorrent
WinSCP

OpenOffice

(no need to list these, it’s the whole sheebang)

Utilities

7-Zip
Audacity
BonkEnc
CCleaner
DeepBurner
ClamWin
CloneSpy
Driver Magician
Floola
Foxit Reader
FSViewer
GnuCash
KeePass
Launchy
MagicDVDRipper
MD5 check sums
PhotoRec
ProduKey
RemoveDrive
Restoration
ShellMenuView
Stellarium
SyncToy
TestDisk
TightVNC
TrueCrypt
VirtuaWin
VirtualDub
VLC Media Player
WinDirStat
WipeDisk

Also SDelete and PsTools which I just use from the command line. That’s some 600MB of software! The only one of these which isn’t a straight download is Skype where I took the skype.exe from the desktop install, ran it through the UPX executable packer, and dumped it on the thumb drive. It’s 12MB and it’s version 3.0.0.198 but it works fine. It requires the following command line parameters:

/datapath:"Data" /removable

so that folder needs setting up …

All of the above programs are installed to the Program Files directory.

The Data folder has a few sub-folders: Backup; KeePass; misc; music; avatars; pictures; scripts; and work. It also has a .tc TrueCrypt file of approximately 128MB. The TrueCrypt file has a folder structure below it that, once mounted as another drive, can hold anything that should be encrypted. You can also have a Program Files directory here for any programs that may have a lot of write cycles to the flash memory, the advantage being that they will be run exclusively from memory once the TrueCrypt drive is mounted. As it works as a drive you could even have an SVN repository here. In theory I could put everything into a TrueCrypt file but then I’d always need admin rights on a machine before I could use my thumb drive and I’d have to enter that really long, complicated passphrase every time I plugged it in.

The music, pictures and work folders are usually empty, only being used for occasional transport duties. Backup is used almost exclusvely for my FEBE backups to transport Firefox extensions from one machine to another. Downloads tends to have just a couple of small utilities that I want on any machine, like GNotify, and otherwise works as another temporary transport folder.

Multiple partitions on a USB key drive

I’m creating multiple partitions on a USB Key (or Flash or Thumb) Drive. Data is shared between partitions and I’ll be able to boot any partition from the drive.

The drive in question is a SanDisk Cruzer Micro Skin 8GB. SanDisk don’t have a utility for flipping the removable bit on the drive and without that XP will not allow the drive to be partitioned. After much reading (the following threads were extremely useful: 911cd.net, msfn.org) and experimenting I’ve finally managed using the Hitachi Microdrive Filter (xpfildrvr1224_320) and adding the correct registry entry to the cfadisk.inf (USBSTOR\Disk&Ven_SanDisk&Prod_Cruzer&Rev_7.01). I then updated the driver (Device Manager > Disk Drives > “USB Drive” > Update Driver) and was able to use Disk Management to create four FAT32 primary partitions.

I used UNetbootin, the Universal Netboot Installer, to install Puppy Linux, UBCD, and Xubuntu. The Portable Apps are described in another blog post. I need to get WinGrub working and then I’ll update this post.