Workflow for Data Analysis - Scott Long
Planning, organizing and documenting :: Using Stata :: Stata Automation :: Names and labels :: Data cleaning
Analysis and presentation :: Backing up and archiving files :: Hardware and software for efficient workflow

Workflow home What's new? Additions by chapter Downloading Stata files My hardware & software Reader's comments Reader's stories Quotes Getting help Disclaimer Home

Hardware and software that I use

The tools you use can make a tremendous difference in your workflow. Here are comments on some of the hardware and sofware that I use or no longer use. No guarantees are implied! Software and hardware change rapidly, so do the tools I use. If you have suggestions of other tools, please let me know.

Text editor · File manager · Hard drives · Backup utilities · Graph utilities · Print utilities · Help

Hard drives

Portable drives: I use a portable drive as the primary location for my work, using the hard drives on my work and home computers to back up the portable drive. That way, I have all of the files I need when I move between computers. Since I back up the portable each time I log off a computer, I don't have to worry about losing or dropping the drive. Recently, I have started using Foldershare to sync files I am actively using on my portable drive to other computers. This means I don't need to carry the portable drive with me every time I change computers.

There are lots of choices. I currently use LaCie's Rugged All-Terrain. It includes both USB and Firewire, seems rugged, and is ugly enough that it is hard to forget it. I avoid portable drives that are hard to grasp, since I hate dropping hard drives. Always eject the drive, don't just unplug it.

External drives: I use several external drives to archive my work. Drive go bad, so I rotate backups among several drives from different manufacturers.

Firewire and Vista: I find that Firewire 400 and Firewire 800 work terribly in Vista running on a MacPro with Bootcamp. USB 2.0 is much faster. Plus, I haven't found an eSATA card that works in a Mac Pro. Oddly, Apple includes two eSATA ports inside the Mac Pro, but there is no easy way to get to them. .

Backup utilities

A good backup utility makes it easier to backup and archive your files. Once you configure a backup utility, a few clicks is all it takes to back up your data.

Allsync has a nice balance between ease of use and power. Plus, if you have a question or problem, technical support is good. Here are a few "tricks" I find useful:

1) Selecting a specific external drive. You can use a drive's volume label when specifying a folder. If you specify a drive as, for example, E:\, Allsync will always look for the E:\ drive. But, with external drives the drive letter can change so that sometimes it is assigned, for example, E:\, but the next time it is F:\. To avoid problems where Allsync can't find the right drive, you can designate the drive as: %<letter>:<volume name>%\ For example, %E:EX18_AT160%\ indicates I want to use the drive with the volume name EX18_AT160 even if it is not the E: drive.

2) Selecting files in folders that end with +: If a folder name end's with the character "+", the files in that folder are posted. For example, e:\Text+\ indicates that the files in this folder are posted and should no longer be changed. You can tell Allsync to backup all files in folders that end with + as follows:

a) In the proporties of a profile, select Filters, Files from the menu.
b) Add a file Filter Type: Inclusive
c) In that filter include: *\*+\*

SyncToy: is a free backup utility from Microsoft. It works quite well if your backup needs are simple. But, I have reservations about recommending for backup something with "toy" in the name.

File manager

One of the great mysteries is why billion dollar companies do not write better file managers than those written by a single person working part time. In my opinion, the best file managers are those based on Norton Commander (NC, 1986-1998) written by John Socha and released by Peter Norton Computing. The fundamental idea is to two have two panels that allow you to move, delete, compare, etc. from one to the other. There are dozens of NC inspired file managers (see Norton Command in Wikipedia).

For Windows, I highly recommend Total Commander (originally Windows Commander until Microsoft threatened to sue). It includes features to zip files, bit compare files and directories, lauch programs, and much more. In Mac, I use Disk Order which is more elegant than Total Commander, but has fewer features. Some users report it is still a bit buggy.

While Total Command and Disk Order contain many utilities for file management, there are two additional utilities that I used regularly (both overlap with what TC does). CloneCleaner provides tools for searching and eliminating duplicate files. This can be very handy in deleting duplicate files in your \- Snapshots folder, etc. RED (Remove Empty Directories) is freeware and works very well at deleting empty directories. (NB: Naming directories are removed by RED since they are often empty).

Text editor

Ultraedit is a great text editor. Here is a syntax file for use with UltraEdit. Please check the UltraEdit documentation for information on how to install the syntax file. Ultracompare is very useful when looking for difference between two versions of a do-file or ado-file (i.e., the one that works and the one that does not!). IDM has announced they are developing Ultraedit for LINUX and Mac. If I can find a Mac version of Scientific Word, I'm ready to move to Mac OS X!

Graph viewers, editors, converters, and databases

Graph viewer: In Windows, IrfranView is a fast, freeware file viewer that includes screen capture and format conversion. It also has a thumbnail viewer that allows you to see many files at once and select the one you want to enlarge.

Screen capture and MUCH more: SnagIt started out as a simple program to capture the image on your screen (i.e., a supercharged PrtScn key). It is now a power program to capture all or part of your screen, edit graphs, convert graphs among formats, capture web sites, and more. I find it extremely useful for simple edits of graphs used in papers or teaching.

Major graphics work and web design: You can't beat Adobe Illustrator, Fireworks, Photoshop, etc. These programs come bundled with Bridge which has become a very good file viewer.

Graph database: Microsoft's Media Expressions is good database for graphic files. This program was called IView before Microsoft bought it.

Printing utilities

FinePrint prints two pages or four pages of output on a single page using any Windows compatible printer. Saves lots of paper.

Your suggestions

Please send me information about your favorite hardware and software.

© 2008 J. Scott Long    
The Workflow of Data Analysis - Principles and practice of effective data management and analysis.