Guild of Project Controls: Compendium | Roles | Assessment | Certifications | Membership

Tips on using this forum..

(1) Explain your problem, don't simply post "This isn't working". What were you doing when you faced the problem? What have you tried to resolve - did you look for a solution using "Search" ? Has it happened just once or several times?

(2) It's also good to get feedback when a solution is found, return to the original post to explain how it was resolved so that more people can also use the results.

Evaluating Software

55 replies [Last post]
Thu, 2003-07-24 10:58
Forum Guest
User offline. Last seen 2 years 28 weeks ago. Offline
Joined: 28 Jan 2009
Posts: 1
Groups: None
Hello,

We been approached by several software vendors who have asked if we can find out more about how a planner values their software, easy of use, shortfalls and functionality etc.

We thought the best way to do this (unless anyone has a better suggestion) is to formulate a list of simple questions that can be given marks out of ten. Several examles could be...

Is it priced right in the market
Marks out of Ten Graphical Reports
Ease of Generating Custon Graphical Reports
Marks out of Ten for Tabular Reports
Ease of Developing Customised Tabular Reports
Usefulness of Software Manual
Usefulness of Softate Support
Is Software Support Good Value
etc etc

Would it be beneficial to list questions that relate to specific planning issues (operation of software to develop network etc) like...

Ease / Speed, Configurization of Time Analysing
WBS Functionality, Ease of Sort Selection
Ease of Custom Layout Saving
Activity Bar Customisation (Colours, Necking, Targets)

If anyone is interested, we would appreciate a volunteer to draw up a list and collate responses here and come up with a concensus of the questions we should ask.

Ideas please...
Top

Replies

Tue, 2003-09-30 23:47
#1
Forum Guest
User offline. Last seen 2 years 28 weeks ago. Offline
Joined: 28 Jan 2009
Posts: 1
Groups: None
Hello PPers,

Please also refer to the following forum posting that should be read in conjunction with this discussion topic.

Please follow this link in order to explore how we could / should implement such a software survey.

Thanks.
Top
Mon, 2003-09-29 10:00
#2
Bernard Ertl
User offline. Last seen 9 years 19 weeks ago. Offline
Joined: 20 Nov 2002
Posts: 757
Groups: Petro-Chem / Oil & Gas, TURNAROUND & SHUTDOWN (T/A & S/D)
Thanks David,

I imagine there would be some human editor maintenance required to enter any "other" fields into database fields that could be searched/sorted/compared against the others. I am not sure if the PP Team was looking to add that to their task list. Of course, having some method for recommending a software to be added to the survey would be a good idea. Perhaps we could also list vendors/software that declined to participate (to prevent wasting time and resources)?

PP Team,

Thanks for all the work you do!

Bernard Ertl
InterPlan Systems Inc. - Project Management Software, Project Planning Software
Top
Mon, 2003-09-29 04:44
#3
Forum Guest
User offline. Last seen 2 years 28 weeks ago. Offline
Joined: 28 Jan 2009
Posts: 1
Groups: None
Hello Fellow PPers,

Country Bias / Location:
We will be able to store the country from which the survey is being taken and as a result we can see and compare what country-by-country differences occur in the same way that we can make comparrisons between packages.

Scope of Software:
We are in a position to include (or otherwise) any software package that we see fit (arent we?). We have spoken with a "majority" of vendors, all of which are keen to be included and wish to be given a chance to see how we will structure the survey. We can approach any vendor and give them the opportunity to offer their requirements and/or opinions.

Progress:
We have spent a few hours here, knocking up some draft scripts to show how the survey could be undertaken. This is being done in parralel with the good work you guys are doing in identifying the topics / questions that we will pose.

We will get these draft scripts on a test-area very shortly and from there we can make some solid progress with regard to the mechanics / procedures of how we allow people to carry out the survey.

I hope some of the above helps?
Top
Mon, 2003-09-29 04:32
#4
David Bordoli
User offline. Last seen 8 years 4 days ago. Offline
Joined: 8 Apr 2002
Posts: 416
Groups: Building Information Modelling
Apologies for missing the deadline. But I think you have missed a trick. Unless the form has changed you seem to have a restricted list of PM Software available for comment (this was the fault of the original survey on PP). So, the least you can do is to add ‘Other – please name’ to the bottom of your list.

Also, is the driving force behind this survey the US? If so then I think the data will be skewed towards the software used there. For instance, in the UK surveys over the last 20 years have produced very different results from those in the UK. In the most recent of surveys (even including the one I undertook 10 years ago) the US was dominated by Primavera and the UK dominated (in construction at least) by Asta’s Powerproject/Teamplan. As far as I could see the structure of the survey would not allow comment to be made on unlisted software. If it is to be a decent survey cognisance has to be given to those that have chosen not to get in touch with you if you know they exist (is Artemis another example)

Again, my apologies for coming late into this and even maybe having misinterpreted what you are doing. I have been working hard and away from my office and haven’t had the opportunity to log on for some time. No criticisms intended.

Regards

David
Top
Thu, 2003-09-25 20:57
#5
Jacques Spilka
User offline. Last seen 19 years 41 weeks ago. Offline
Joined: 14 Sep 2003
Posts: 11
Groups: None
Never mind!

Thanks Bernard.

JAS
Top
Thu, 2003-09-25 10:41
#6
Bernard Ertl
User offline. Last seen 9 years 19 weeks ago. Offline
Joined: 20 Nov 2002
Posts: 757
Groups: Petro-Chem / Oil & Gas, TURNAROUND & SHUTDOWN (T/A & S/D)
Jacques,

Mark was referring to Abdullah Bani Ali`s survey (linked in my previous post).

Also, the PP survey (linked previously) has not been updated to incorporate any of the suggestions mentioned here yet. We will be working on that shortly.

Bernard Ertl
InterPlan Systems Inc. - Project Management Software, Project Planning Software
Top
Thu, 2003-09-25 10:28
#7
Jacques Spilka
User offline. Last seen 19 years 41 weeks ago. Offline
Joined: 14 Sep 2003
Posts: 11
Groups: None
I disagree. IMHO the survey is not too long.

Ive also noted that it is lacking the suggestion from Bernards 16 Sep 2003 at 08:36 post. These would be a valuable addition.

JAS
Top
Thu, 2003-09-25 09:43
#8
Bernard Ertl
User offline. Last seen 9 years 19 weeks ago. Offline
Joined: 20 Nov 2002
Posts: 757
Groups: Petro-Chem / Oil & Gas, TURNAROUND & SHUTDOWN (T/A & S/D)
I took a quick look at Abdullah Bani Ali`s Project Management Software Effectiveness Survey and it appears to have a slightly different focus than what we are aiming for here.

I think we could probably incorporate some aspects of
  • Section I - Frequent of Use (sic)
  • Section I - Extent of Use


Bernard Ertl
InterPlan Systems Inc. - Project Management Software, Project Planning Software

Top
Thu, 2003-09-25 01:55
#9
Mark Lomas
User offline. Last seen 15 years 47 weeks ago. Offline
Joined: 27 Jun 2001
Posts: 177
Groups: None
Hell, thats what we needed, a doctoral research student ! The size of that survey !

Seriously, the posted survey is about the use of PM software and ours is about comparative functionality of Planning software. But there may be things in there that could help our survey.

Lets review it when we close comments, i.e. today.

Mark
Top
Mon, 2003-09-29 01:26
#10
Mark Lomas
User offline. Last seen 15 years 47 weeks ago. Offline
Joined: 27 Jun 2001
Posts: 177
Groups: None
Guys,

thanks for your input. The survey is now closed to comment and is being edited. Watch this space.
Top
Wed, 2003-09-24 08:19
#11
Bernard Ertl
User offline. Last seen 9 years 19 weeks ago. Offline
Joined: 20 Nov 2002
Posts: 757
Groups: Petro-Chem / Oil & Gas, TURNAROUND & SHUTDOWN (T/A & S/D)
OK folks, tomorrow is the deadline for suggestions. Please take a minute to review the proposed survey (Proposed Planning Planet Survey) and offer your input before it is too late.

Bernard Ertl
InterPlan Systems Inc. - Project Management Software, Project Planning Software
Top
Tue, 2003-09-23 11:30
#12
Jacques Spilka
User offline. Last seen 19 years 41 weeks ago. Offline
Joined: 14 Sep 2003
Posts: 11
Groups: None
Bernard,

I think you are correct in your assessment that we are getting into a subjective area. Since I initially brought up the issue of effectiveness I am now willing to remove it, if no one else minds. This would allow us to focus on comparing package features.

Is there a new version of the Proposed Planning Survey as well as a list of the packages to be evaluated?

JAS
Top
Tue, 2003-09-23 09:48
#13
Bernard Ertl
User offline. Last seen 9 years 19 weeks ago. Offline
Joined: 20 Nov 2002
Posts: 757
Groups: Petro-Chem / Oil & Gas, TURNAROUND & SHUTDOWN (T/A & S/D)
In order to maintain the original aim of this survey - to be able to compare different systems against each other - I would prefer to maintain the 1-5 & N/A rating system.

While measuring effectiveness is a great idea, the latest proposed format really does not offer any meaningful basis for comparing two (or more) different systems. IMHO, there are too many external variables that affect a planners/companys implementation. I think the focus should remain on the capabilities / features of the software.

Perhaps we can expand upon the functionality survey question and break it down into subitems:
  • flexibility
  • usefulness
  • robustness
  • etc.

Alex (and everyone), we are currently generating the format for the survey. We invite you to share your thoughts. Please offer specific suggestions.

Bernard Ertl
InterPlan Systems Inc. - Project Management Software, Project Planning Software
Top
Tue, 2003-09-23 08:24
#14
Jacques Spilka
User offline. Last seen 19 years 41 weeks ago. Offline
Joined: 14 Sep 2003
Posts: 11
Groups: None
Mark,

That looks like a good solution for measuring effectiveness.

btw, Ive contacted Realizations, the company that markets Concerto. They seem interested in participating.

Regards,

JAS
Top
Tue, 2003-09-23 00:27
#15
Mark Lomas
User offline. Last seen 15 years 47 weeks ago. Offline
Joined: 27 Jun 2001
Posts: 177
Groups: None
Jacques,

on effectiveness we could either give a "tick all that apply" list of statements, eg:
    has improved my effectiveness as a planner
    has improved my departments throughput
    has improved my companies profitability
    etc

or
give a 1-5 and n/a grading option against a similar series of statements.

Alex,

thanks. Any comment on the survey structure so far ?

All,

we may also want to consider a best feature / worst feature type-in box.

Mark
Top
Mon, 2003-09-22 23:32
#16
Alex Wong
User offline. Last seen 11 years 4 weeks ago. Offline
Joined: 12 Feb 2003
Posts: 874
Groups: TILOS
Team,

I would love to be involve in the survey.

Cheers

Alex
Top
Mon, 2003-09-22 08:08
#17
Bernard Ertl
User offline. Last seen 9 years 19 weeks ago. Offline
Joined: 20 Nov 2002
Posts: 757
Groups: Petro-Chem / Oil & Gas, TURNAROUND & SHUTDOWN (T/A & S/D)
Mark, Jaques, et. al.,

Ease of Use is already included in the survey with two subtopics (though they need to be indented for clarity). Do most people think that Ease of Use (and/or usefulness) should be a subitem of every listed feature instead?

Bernard Ertl
InterPlan Systems Inc. - Project Management Software, Project Planning Software
Top
Mon, 2003-09-22 08:59
#18
Jacques Spilka
User offline. Last seen 19 years 41 weeks ago. Offline
Joined: 14 Sep 2003
Posts: 11
Groups: None
So there are three things we want to measure then:
1) Functionality
2) Ease of use
3) Usefulness/effectiveness

Certainly 1 and 2 above could be measured for each feature. For item 3 it may be more productive to assess the entire PM package or major components of the package (e.g. resource leveling, portal...) to determine relevance.

Also a clear definition of usefulness and/or effectiveness will be required. What may be useful for an individual is not always effective for the organization. We should only measure one or the other and ensure that the respondents understand the meaning of the measure.
Top
Mon, 2003-09-22 01:04
#19
Mark Lomas
User offline. Last seen 15 years 47 weeks ago. Offline
Joined: 27 Jun 2001
Posts: 177
Groups: None
Thanks Jaques,

When we get around to restructure the survey, based in feedback herein, should we consider:
    Ease of Use
    Usefullness

as the two main headings ?

Ease of use then splits into:
    Input
    Process
    Output

containing all the information we have gathered to date.

Usefullness can then follow Jaquess suggestions, either on a personal level (a planner may feel it makes him / her more effective) or a corporate level (either with or without data to back it up).
Top
Wed, 2003-09-17 14:30
#20
Jacques Spilka
User offline. Last seen 19 years 41 weeks ago. Offline
Joined: 14 Sep 2003
Posts: 11
Groups: None
Thats pretty good for someone out of his area of expertise. Heres my suggestion:

Collaboration engine:
Project Portal (proprietary)
MS Outlook
Lotus Notes

Navigability:
Click to view upstream tasks
Click to view downstream tasks
Click to view tasks assigned to same resource
Click to view all tasks controlled by task manager

Roll-Up facilities:
Costs/Accounting
WBS levels

Portfolio Management:
Ability to rank projects by priority
Ability to perform "what-if" analysis impact on all projects in the portfolio
Ability to synchronize task priority with project portfolio priority

Effectiveness:
The use of this tool has increased project throughput (more projects per year without increasing resources)
The use of this tool has eased the PM administrative burden
The use of this tool has improved project delivery (on-time, on budget, in scope)

One of the problems that I foresee is that respondants may not know how to evaluate some of these (e.g. project throughput)

JAS
Top
Tue, 2003-09-16 11:18
#21
Bernard Ertl
User offline. Last seen 9 years 19 weeks ago. Offline
Joined: 20 Nov 2002
Posts: 757
Groups: Petro-Chem / Oil & Gas, TURNAROUND & SHUTDOWN (T/A & S/D)
Jacques, good ideas. Can you lay them out in a more specific fashion? Is the following what you had in mind (for the 1-5 rating system with N/A options)?

Collaboration
  • (Proprietary) Project portal
  • Microsoft Exchange integration
  • Lotus Notes integration
  • Ease of use
  • Navigability
    • Up stream task
    • Downstream task
  • Roll-Up Facilities
    • costs / accounting
    • timelines
    • Portfolio management

This is not my area of expertise, so the above may not make sense - please offer suggestions.

Bernard Ertl
InterPlan Systems Inc. - Project Management Software, Project Planning Software
Top
Tue, 2003-09-16 09:46
#22
Jacques Spilka
User offline. Last seen 19 years 41 weeks ago. Offline
Joined: 14 Sep 2003
Posts: 11
Groups: None
This sounds very good. May I suggest we add a Collaboration section (e.g. Project portal, Exchange or Lotus Notes integration...)?
It should include some measure of ease of use and navigability (up stream task, down stream tasks)
Also some sense of the roll-up facility (costs, timelines. Finally portfolio management would seem to be an important feature, particularly for multi-project matrix organizations.
Top
Tue, 2003-09-16 09:36
#23
Bernard Ertl
User offline. Last seen 9 years 19 weeks ago. Offline
Joined: 20 Nov 2002
Posts: 757
Groups: Petro-Chem / Oil & Gas, TURNAROUND & SHUTDOWN (T/A & S/D)
IMHO, the current survey as proposed is adequate to allow comparing of all manner of project planning, scheduling and management tools. The 1-5 rating system with the inclusion of a N/A option should allow direct comparisons of any feature via a database search on an apples to apples basis. The key is to ensure that the survey is asking for all the important features (and at an appropriate level of definition to be meaningful).

In reviewing the survey again, I think the software reviewed question should be a pick list instead of a fill in the blank to ensure proper categorization of the results.

Also, I would like to suggest the following changes
  • progress tracking - break out the different possible measures of percent complete (time, effort and cost) as sub-items
  • data sharing - break out import and export as sub-items
  • planning - add sub-items for more definition:
    • pre-defined coding structures
    • user defined coding structures
    • organization (work order / WBS / etc.)
    • data re-use facilities (templates / database / estimating engine)
  • scheduling - add sub-items for more definition:
    • critical path scheduling
      • speed of calculations
      • PDM interface
      • ADM interface
    • critical chain scheduling
      • speed of calculations
      • PDM interface
      • ADM interface (are there any ADM style CC systems?)
    • calendar based scheduling (no logic network support)
  • cost estimating - add sub-items for more definition:
    • detail
    • summary
  • reporting - add sub-items for more definition:
    • pre-defined (hard coded) reports
    • ad-hoc (user defined) reporting
    • presentation
    • readability
    • useability
If sub-items are defined, perhaps the parent item should not be a user defined field - it should be calculated as an average of the sub-item responses.

Also, all listed applications in the survey should include a version number (ie. P3e/c vs. 3.5 or whatever) as survey results may change across different versions of the same product.

Hopefully this will spark some additional ideas...

Bernard Ertl
InterPlan Systems Inc. - Project Management Software, Project Planning Software
Top
Tue, 2003-09-16 09:04
#24
Jacques Spilka
User offline. Last seen 19 years 41 weeks ago. Offline
Joined: 14 Sep 2003
Posts: 11
Groups: None
Can you provide a clear definition of PLANNING package to avoid confusion?
Top
Tue, 2003-09-16 05:39
#25
Mark Lomas
User offline. Last seen 15 years 47 weeks ago. Offline
Joined: 27 Jun 2001
Posts: 177
Groups: None
I had presumed that all the packages we were listing were PLANNING packages. If, as Nestor points out, we widen to non-planning packages then we cant compare.

If we stick to planning packages only (for now), does the survey give enough depth ? We may wish to break out under "functionality" to include progress spotlights, EVA, resource loading, filtering, etc., etc.. Thaoughts on this are possibly what the vendors may be after.
Top
Tue, 2003-09-16 05:09
#26
Soheil Jafari
User offline. Last seen 3 years 35 weeks ago. Offline
Joined: 3 Jul 2001
Posts: 29
Groups: Petro-Chem / Oil & Gas
Nestor,
I agreed with you, lets talk and share our information based on systematic regulations and after that make a suitable conclusion .
Top
Mon, 2003-09-15 09:50
#27
Bernard Ertl
User offline. Last seen 9 years 19 weeks ago. Offline
Joined: 20 Nov 2002
Posts: 757
Groups: Petro-Chem / Oil & Gas, TURNAROUND & SHUTDOWN (T/A & S/D)
Nestor,

Have you reviewed the proposed survey? Can you offer specific recommendations in line with your thoughts? Please post a sample of what you propose.

Bernard Ertl
InterPlan Systems Inc. - Project Management Software, Project Planning Software
Top
Mon, 2003-09-15 07:24
#28
Nestor Principe
User offline. Last seen 20 years 6 weeks ago. Offline
Joined: 17 Jul 2003
Posts: 19
Groups: None
Dear All,

It appear to me that the task of comparing these software packages is becoming complicated. In fairness, we want to make sure we are comparing apple to apple and not car to a truck.

Maybe, we could make the comparison in two parts. The first part comparing the basic functions of the planning softwares and the second part comparing the specialist functions. We need to categorize the the planning softwares in the second part.

Just an idea.

Regards,

Nestor
Top
Sun, 2003-09-14 16:29
#29
Forum Guest
User offline. Last seen 2 years 28 weeks ago. Offline
Joined: 28 Jan 2009
Posts: 1
Groups: None
"Risk/Uncertainty management" sounds like a good category to add.
My suggestion was also to meant to include measuring how effective the introduction or use of PM software was to the implementing organization. Can this be measured? Is there a common metric across organizations? A measure of total project days per employee per year. Measuring the total departmental project completion rate per employee over time would allow you to see the effectiveness of the PM system.
This may be a moot point however as I doubt most organizations would have any historical information on this measure.
Top
Sat, 2003-09-13 10:02
#30
Bernard Ertl
User offline. Last seen 9 years 19 weeks ago. Offline
Joined: 20 Nov 2002
Posts: 757
Groups: Petro-Chem / Oil & Gas, TURNAROUND & SHUTDOWN (T/A & S/D)
Would adding a line item for "Risk/Uncertainty Management" to the Project Functionality categories cover it?

Perhaps the PP team can contact Realization to see if they would like to participate in the survey?

Bernard Ertl
InterPlan Systems Inc. - Project Management Software, Project Planning Software
Top
Sat, 2003-09-13 00:45
#31
Forum Guest
User offline. Last seen 2 years 28 weeks ago. Offline
Joined: 28 Jan 2009
Posts: 1
Groups: None
Is it too late to add a vendor to the evaluation list?
If not, I propose we add Concerto from www.realization.com.

Concerto adds the dimension of managing uncertainty and appears to have a very good track record in terms of increasing project througput.

Does this argue for adding an effectiveness component to the evaluation criteria?
Top
Fri, 2003-09-12 01:02
#32
Forum Guest
User offline. Last seen 2 years 28 weeks ago. Offline
Joined: 28 Jan 2009
Posts: 1
Groups: None
Gents,

I have changed the main front page as suggested. Lets see what help we get. I have been in Hong Kong on business, I will review your most-impressive question sheet!
Top
Thu, 2003-09-11 01:09
#33
Mark Lomas
User offline. Last seen 15 years 47 weeks ago. Offline
Joined: 27 Jun 2001
Posts: 177
Groups: None
James, can you also put an alert on the homepage that we are requesting comment on this survey. It may prompt more people to respond. Thanks.

Mark
Top
Wed, 2003-09-10 22:02
#34
Bernard Ertl
User offline. Last seen 9 years 19 weeks ago. Offline
Joined: 20 Nov 2002
Posts: 757
Groups: Petro-Chem / Oil & Gas, TURNAROUND & SHUTDOWN (T/A & S/D)
Yes, two weeks is more than reasonable for everyone who is interested to have chance to offer their input.

Folks, opportunity is knocking. If you want a tool to compare software packages, act now to ensure you get something you can use.

Bernard Ertl
InterPlan Systems Inc. - Project Management Software, Project Planning Software
Top
Wed, 2003-09-10 21:17
#35
Mark Lomas
User offline. Last seen 15 years 47 weeks ago. Offline
Joined: 27 Jun 2001
Posts: 177
Groups: None
Thanks Bernard,

May I now suggest we set a two week limit for comment. By close on Thursday 25 Sep 03 ?

Following that we can offer to the vendors for comment before starting to run surveys.

Mark
Top
Wed, 2003-09-24 09:12
#36
Jacques Spilka
User offline. Last seen 19 years 41 weeks ago. Offline
Joined: 14 Sep 2003
Posts: 11
Groups: None
Looks good. Here are my comments:

Under "What software do you also have experience with"
PLEASE ADD:
- Concerto - Realization Technologies, Inc.

Under "Stand Alone Project Functionality" and "Enterprise Project Functionality"
PLEASE ADD:

- Navigability:
Drill through to upstream tasks
Drill through to downstream tasks
View tasks assigned to same resource
View all tasks controlled by task manager

- Portfolio Management:
Projects by priority ranking mechanism
Task priority ranking mechanism
Ability to perform "what-if" analysis impact on all projects in the portfolio

- Critical Chain Buffer Management

Please let me know if you need clarification.

JAS
Top
Wed, 2003-09-10 06:07
#37
Bernard Ertl
User offline. Last seen 9 years 19 weeks ago. Offline
Joined: 20 Nov 2002
Posts: 757
Groups: Petro-Chem / Oil & Gas, TURNAROUND & SHUTDOWN (T/A & S/D)
Great job Mark! I have posted the Word document to our server for all to review: Proposed Planning Planet Survey Anyone who is interested should feel free to offer comments regarding the survey either by posting here in this thread or e-mailing me (bernarde at my company domain), Mark (through the PP member link) or the PP team. Please do not e-mail us with your answers / completed survey at this time! We are seeking comments to refine the survey questions. Please help ensure that this survey will be a useful tool for everyone to compare the relative strengths and weaknesses of available commercial project planning and project management software. Bernard Ertl InterPlan Systems Inc. - Project Management Software, Project Planning Software
Top
Tue, 2003-09-09 20:37
#38
Mark Lomas
User offline. Last seen 15 years 47 weeks ago. Offline
Joined: 27 Jun 2001
Posts: 177
Groups: None
Bernard,

I have added to your question set (from this thread) and sent it to you and James by email. I do not have an appropriate site to post it to - grateful if you (or James) would.

We can then get it refined, before checking that it will give the desired results.

Mark
Top
Tue, 2003-09-09 09:12
#39
Bernard Ertl
User offline. Last seen 9 years 19 weeks ago. Offline
Joined: 20 Nov 2002
Posts: 757
Groups: Petro-Chem / Oil & Gas, TURNAROUND & SHUTDOWN (T/A & S/D)
PP Team,

Good news! I was beginning to wonder if this project was dead.

I have already offered some input that may help form a base for the study in my post dated 28 July in this thread. I am willing to help where I can.

Bernard Ertl
InterPlan Systems Inc. - Project Management Software, Project Planning Software
Top
Tue, 2003-09-09 02:08
#40
Forum Guest
User offline. Last seen 2 years 28 weeks ago. Offline
Joined: 28 Jan 2009
Posts: 1
Groups: None
Hi there,

Objectives of the Exercise
To develop and investigate the differences (strengths and weaknesses) of various Project Planning software packages. The primary aim being to be able to compare the packages wherever possible.

Tasks and Deadlines
We ideally need someone in the PP group to take it on themselves to draft a set of questions. The simplest thing, in order to be able to compare these packages, is to offer a picklist of set answers wherever possible.

In my humble opinion it would be good to define tasks and deadlines once one of the PP member-group has come forward to take the lead to collate answers/opinions and write out useful questions - perhaps from some forum QandA etc.

Some consideration needs to be given to the fact that some packages have specialist objectives - so we should endeavour to avoid penalising them for not offering all of the usual common planning-tool functions etc.

What do you think?
Top
Tue, 2003-09-09 01:51
#41
Mark Lomas
User offline. Last seen 15 years 47 weeks ago. Offline
Joined: 27 Jun 2001
Posts: 177
Groups: None
James,

A step back. Lets plan what we are to do:

Given that you have collated all the responses from vendors, can you define the objectives of the exercise ? When we have defined the objectives we can begin to structure the survey / questions set to give the required output. Can we also agree some tasks and deadlines for the process before we issue the survey / questions set to the vendors ?

A salutary lesson, which Im sure we are all aware of - dont charge off and start doing the work without adequate planning !
Top
Tue, 2003-09-09 01:29
#42
Forum Guest
User offline. Last seen 2 years 28 weeks ago. Offline
Joined: 28 Jan 2009
Posts: 1
Groups: None
We have had a good response from many suppliers.

It is now only a case of finding the time to formulate a list of questions for us all to answer - thereby generating the results.

We have also been asked to allow the software people the option to reqview our set of questions before we go live with the survey.

We have had several emails suggesting that the survey should offer the voter the choice of which packages he / she wishes to review. Following on from these, options would be provided in order to grade / score each (comparatively) against each other.

Sounds like an interesting thought.

Anyone of you PPers with to volunteer to write the questions using input from the other members (who are interested) ?
Top
Wed, 2003-07-30 04:34
#43
Forum Guest
User offline. Last seen 2 years 28 weeks ago. Offline
Joined: 28 Jan 2009
Posts: 1
Groups: None
Nope, no Artemis - they have not replied to us yet.

However, we will still include them, and they cant say we did not offer / request their assistance!
Top
Wed, 2003-07-30 04:25
#44
Forum Guest
User offline. Last seen 2 years 28 weeks ago. Offline
Joined: 28 Jan 2009
Posts: 1
Groups: None
Wot no Artemis?
Top
Mon, 2003-07-28 18:42
#45
Forum Guest
User offline. Last seen 2 years 28 weeks ago. Offline
Joined: 28 Jan 2009
Posts: 1
Groups: None
That is a very complete list of software on your site! Might I suggest for our purposes, that we also break-out the software by major version where appropriate?

MS Project 98 is far different from MSP 2000 and that is far different from the MSP Enterprise version available today. Lumping these together would confuse, rather than clarify the responses.
Top
Mon, 2003-07-28 17:06
#46
Ronald Winter
User offline. Last seen 3 years 6 weeks ago. Offline
Joined: 4 Jan 2003
Posts: 928
Groups: None
You have a small problem here. Do I understand that we are seeking the opinion of only those people who have already bought the software and not of people who have experience in all of the soft wares being reviewed?

Don’t you think that someone who bought a particular piece of software would be biased toward that software, even without trying the other brand? Otherwise, why did they purchase it? In addition, they may think that ‘everybody does it this way’ because of their limited experience and not know that a superior feature exists in another package?

Perhaps each rating should be accompanied by a list of software that that person has used. Then their comments can at least be taken in context.
Top
Mon, 2003-07-28 10:50
#47
Bernard Ertl
User offline. Last seen 9 years 19 weeks ago. Offline
Joined: 20 Nov 2002
Posts: 757
Groups: Petro-Chem / Oil & Gas, TURNAROUND & SHUTDOWN (T/A & S/D)
IMO, some background on the poll participants is necessary to qualify their votes. The Planning Planet membership is spread out across a range of industries and project management is handled differently in each. Ill propose the following comparative analysis questions (feel free to add, remove or amend suggestions as you wish):


  • Set of Questions (marks out of ten)

    • Participant background

      • What industry are you in? [Multiple Choice - Engineering, Construction, IT, Rail, Petrochem/Refining, etc.]
      • How many years of experience do you have with this software? [Multiple Choice - Limited, 1, 2, 3-5, 6-10, 11+]
      • What is your current role? [Multiple Choice - Owner/Employee, Consultant, Affiliate/Reseller]

    • Software Review [all ?s = ranges/ratings 1-10 with a N/A option]

      • Ease of use

        • Software (user interface)?
        • System (to manage a project)?

      • Documentation

        • Printed?
        • Electronic (help system)?

      • Stand Alone Project Functionality

        • Collaboration?
        • Planning?
        • Cost Estimating?
        • Scheduling?
        • Resource Management?
        • Progress Tracking?
        • Earned Value Analysis?
        • Reporting?
        • Data Sharing (Import/Export facilities)?

      • Enterprise (Multiple) Project Functionality

        • Collaboration?
        • Planning?
        • Cost Estimating?
        • Scheduling?
        • Resource Management?
        • Progress Tracking?
        • Earned Value Analysis?
        • Reporting?
        • Data Sharing (Import/Export facilities)?

      • Vendor Support?
      • Overall Value (for the cost)?




I would prefer to limit participation in the polling to PP members and to limit voting to one time (per product) per member. If new versions are released, a new poll should be added for that release (ie. treat each version of a program as a separate product for the purposes of polling).

All 1-10 votes (excluding any N/A answers) could be averaged to display an overall review rating.

[quote]Free Form Commentary - this is good as it will offer the chance to put our specific concerns, complaints or suggestions across.

If we have a survey with a lot of textual (textual!??) answers, will this make it difficult to form a comparison?[/quote]

Perhaps just one free form box will be sufficient for the participant to address any points which the comparative analysis questions do not address adequately.

Bernard Ertl
InterPlan Systems Inc. - Project Management Software, Project Planning Software
Top
Mon, 2003-07-28 04:54
#48
Forum Guest
User offline. Last seen 2 years 28 weeks ago. Offline
Joined: 28 Jan 2009
Posts: 1
Groups: None
We have a guy called Winston, from Microsoft who wants to see what the "masses" think of their MS Project suite. His response to our request for some questions was that he did not mind what we asked (hmmm?), but was concerned that the process could...

(a) Offer comparrative feedback such that their strengths could be measured against another competitors.

(b) Offer some means of allowing specific, personal feedback from people in the field.

Bernards template is great. Things to consider...

Set of Questions (marks out of ten) - this will allow the some quantitive analysis and feedback, giving the potential to rate one package / tool against another.

Free Form Commentary - this is good as it will offer the chance to put our specific concerns, complaints or suggestions across.

If we have a survey with a lot of textual (textual!??) answers, will this make it difficult to form a comparrison?

Our man at Microsoft (Winston, who does not think any free software will be on the cards) and would prefer a survey in the form of a set of questions (marks 5 or 10). He also wishes to leave the survey running over time and said it should be open to all the site planners.

Do we then go for...

A user free-form text box plus marks out of ten for (a) overall (b) ease of use (c) functionality etc.

OR

A user free-form text box plus marks out of ten for (a) overall (b) WBS functionality (c) network plotting functionaity (d) Text reports etc etc

Appologies for only creating more questions. It would probably be best to run with the MS Project initially, as it is well-known. However, how to we show any results in a comparative format if we do not run the survey for several packages from day one?
Top
Mon, 2003-07-28 01:01
#49
Mark Lomas
User offline. Last seen 15 years 47 weeks ago. Offline
Joined: 27 Jun 2001
Posts: 177
Groups: None
OK, Bernard has outlined a marking / review template. Im sure that is under control.

Next, what is the process PP will employ to carry out reviews ?
Will we spotlight one product per month ? Two ?
How do we decide what to review ?
Who reviews and how many PP reviewers do we use ?
How do we recruit them ?

I suggest we do a trial run. James, please shortlist 3 or 4 of the interested parties so we can decide which one to trial. We should then approach the vendor with our suggested process and results (Bernards template) and get agreement. Once the vendor is happy, we can start.

If it is a review of software existing in the market, we can invite PP users (open invite) to give ther evaluations (based on Bernards template). Give PPers 2 weeks to reply from invite. PP moderators then need to collate (and perhaps edit) the reviews, before we post a consolidated review on PP.

If it is a review of new software, perhaps the vendor can issue, say, 10 trial versions for PP evaluation.

Remember, this is a service PP is providing. PP should get some benefit (freebies from the vendors, to be given away as prizes), and should look at its disclaimers.
Top
Fri, 2003-07-25 11:06
#50
Bernard Ertl
User offline. Last seen 9 years 19 weeks ago. Offline
Joined: 20 Nov 2002
Posts: 757
Groups: Petro-Chem / Oil & Gas, TURNAROUND & SHUTDOWN (T/A & S/D)
[quote]We been approached by several software vendors who have asked if we can find out more about how a planner values their software, easy of use, shortfalls and functionality etc.[/quote]

  1. Planner valuation of software (overall) - This could easily be a generic 1-10 opinion rating question by itself.
  2. Ease of use - Same as above
  3. Shortfalls - Free-form commentary?
  4. Functionality - If we use the question/rating format, the detail question comes into play. The free-form commentary format may be more appropriate.
  5. Excellence - Why not point out strengths as well? Free form comentary as with shortfalls

Some consideration should be given to the date that a review is posted to match up with a particular version of the software. Vendors may improve their systems based upon commentary and release new versions. How will this system maintain relevancy? Clear the database upon new releases?

Bernard Ertl
InterPlan Systems Inc. - Project Management Software, Project Planning Software
Top
Fri, 2003-07-25 10:59
#51
Bernard Ertl
User offline. Last seen 9 years 19 weeks ago. Offline
Joined: 20 Nov 2002
Posts: 757
Groups: Petro-Chem / Oil & Gas, TURNAROUND & SHUTDOWN (T/A & S/D)
[quote]Just what level of detail do we "collectively" want to go to? Who would value that level of detail? [/quote]

Good question Guy!

IMO, detail is going to have an exponential relationship with the number of questions/factors to be asked/reported. Each software package has its own strengths and weaknesses. What about niche products?

Bernard Ertl
InterPlan Systems Inc. - Project Management Software, Project Planning Software
Top
Fri, 2003-07-25 03:49
#52
Guy Hindley
User offline. Last seen 5 years 6 weeks ago. Offline
Joined: 28 Sep 2001
Posts: 91
Groups: None
Great idea.

Lets see what the software vendors feel they offer, and if there are differences between what we want/ need as Planners. We could work on both approaches independantly and then correlate the two and identify any differences, if any.
Top
Fri, 2003-07-25 01:16
#53
Mark Lomas
User offline. Last seen 15 years 47 weeks ago. Offline
Joined: 27 Jun 2001
Posts: 177
Groups: None
May also I suggest we go back to the vendors that have approached PP and ask them what sort of feedback they require.

This is a normal up-front PM activity - define the customers requirements.
Top
Thu, 2003-07-24 12:06
#54
Guy Hindley
User offline. Last seen 5 years 6 weeks ago. Offline
Joined: 28 Sep 2001
Posts: 91
Groups: None
Seems like a good idea. A feature a find many products lack is a good clearly laid PERT of network logic view. To me this is a key feature when creating and building plans form scratch. Without mentioning product names one major vendor makes his money by providing a graphical add on service to virtually all, if not all current planning software packages out there in the market place. Perhaps the details of graphical reporting could be split to take into account: network view, barcharts, resources, wbs/ obs structure drawing etc. Usability/ configurability, etc could all be scored

I feel sure that other Planning Planeters could add to this initial stab of mine.

Just what level of detail do we "collectively" want to go to? Who would value that level of detail?
Top
Mon, 2003-07-28 17:33
#55
Bernard Ertl
User offline. Last seen 9 years 19 weeks ago. Offline
Joined: 20 Nov 2002
Posts: 757
Groups: Petro-Chem / Oil & Gas, TURNAROUND & SHUTDOWN (T/A & S/D)
Ron, the intention with my suggested questions was to allow members to review individual software as they like. Some software will naturally have more reviews posted as I perceive the Planning Planets membership to be skewed in favor of Primavera users.

I tried to account for the bias you mention by including a question covering years of experience with the software being reviewed.

There really isnt any way to remove all bias from an open (and anonymous) survey.

BTW, Ive catalogued about two dozen different Windows based project management systems in my Project Management Software Directory. There are also many web based systems. Are we going to include them all??

Bernard Ertl
InterPlan Systems Inc. - Project Management Software, Project Planning Software
Top