Methodologist’s Toolchest

Methodologit’s Toolchest V. 2.0 (1998) by the Idea Works, Inc. - "The researcher still makes the final decision, but now he or she can be confident that it is an informed one."

I obtained a free demonstration version but it unfortunately has a predefined tutorial. I was not able to work with my own data. And it’s not really a tutorial, it’s a tour.

3. What is the intended audience for the software?

The Methodologist’s Toolchest is for anybody planning a research project. It is designed "to help you eliminate costly errors in funding proposals" by walking you through the construction of a research proposal. It states that researchers writing funding proposals, students designing theses and dissertations, or staff scientists designing in-house research projects will all benefit from it use.

4. What are the range of functions offered?

The program has the following subprograms:

Major sections (checklists) of Peer Review Emulator include:

AIMS – type of study, contributions, objectives, plausibility, testability/falsifiability, importance

Background and Significance – builds from and/or critiques past research, quality of lit review, relevance

Content – domain, assumptions, theoretical perspective, scope/context, units of analysis, population

Relevant Experience – previous studies and results, changes in objectives, intellectual products

Research Design & Methods – data collection procedures, measurement strategies, design review, sampling plan, analysis plans

Ethics and Management – human subjects concerns, schedule, budget, personnel, facilities

First Draft Editor

Overall observations:

5. How would an ERG researcher make use of this software? (Which of its functions will actually be useful?)

Difficult to say, each subprogram has something to offer. For example, I don’t expect you’d use the draft editor, but the peer review checklist may be very useful. Some researchers may appreciate having a tool that tracks your progress, or that organizes boiler plate text. I probably wouldn’t use it for that.

The Ex-Sample subprogram may be a good disciplined way to work out sampling needs, but some may find it to be too constrained. The language seems very "plain english" which would help me. The sensitivity modeling would be quite helpful, I think.


6. How does the software design affect or restrict the research process in general? (Where does it fit? What are its limits?)

MT is a comprehensive research design tool, but it won’t do you research for you. And you really need to do the designing. It just provides a good checklist of questions and design tests, as well as one-stop shopping for definitions and descriptions.

5. What is your overall evaluation of the software package, in terms of both software design and execution, and functionality?

Difficult to say, because I couldn’t really test it. In fact, the demonstration isn’t really a way of demonstrating how the program works. Instead, the demo is just a walk through program features. Unfortunately, to truly test it, you’d need to use it. I suspect that it doesn’t really deliver all that the demo says, that the "interaction" isn’t quite so fluid, and that the expert system isn’t really such a smart "expert". If it were, you wouldn’t have to buy the program to find out, you’d be allowed to work with it as a true demonstration version.