(1) Explain your problem, don't simply post "This isn't working". What were you doing when you faced the problem? What have you tried to resolve - did you look for a solution using "Search" ? Has it happened just once or several times?
(2) It's also good to get feedback when a solution is found, return to the original post to explain how it was resolved so that more people can also use the results.
Dear all,
In the parallel thread I published results of resource a leveling engine comparison, which I did on Microsoft Project, Primavera, Asta Powerproject, and Spider Project. However some people pointed out that my test results may be a bit “colored”, as I used Test Schedules from Vladimir Liberzon (Spider Project). Since Spider came to be by far the best out of my testing, Lord mentioned, that some other manufacture may be able to come up with the specially crafted schedule, where their tool would be better, then the ones of the competition.
Therefore I came to an idea to do a little project to compare resource leveling engines of different tools in a more structured and transparent way.
However to do this, we first need to define rules of the game. Therefore I have drafted an initial version of a document, which I called “Tests description and requirements to test schedules for automatic resource leveling functionality comparison test”.
Therefore I am kindly asking everybody, who is intersted, to do the following:
Regards.
Evgeny
========================================================================
Tests description and requirements to test schedules for automatic resource leveling functionality comparison test
Automatic resource leveling functionality is considered by many to be an essential feature of the scheduling software. However, unlike Critical Path calculation, there is no well-defined algorithm, which would always produce the most optimal result. Therefore different manufactures have a different implementation of resource-leveling algorithms, which often produce different result for the same schedule. Also often specific tool shows better results for one schedule, whilst losing to alternative tool on another schedule.
As a result of several discussions on the world’s largest planning community site ( www.planningplanet.com ), an initiative came up to run a little project to compare resource leveling algorithms of different scheduling tools with each other.
The idea is that selected tools will be tested on the set of Test Schedules, consequently outcomes will be published and analyzed. As a result both users as well as manufactures will be able see strong and weak points of resource leveling of different scheduling tools in specific situations. Hence users will be able to make an informed decision about usage of a specific tool; manufactures will be able to see strong as well as improvement points for their software.
The plan is to from one side make the tests as unbiased as possible and from the other side to allow different manufactures to show the strong point of their tools against competition by coming up with their version of a test schedule (provided it meets the requirements, agreed in this document).
This document will be used to:
GR001 Test schedules shall be only targeted for testing of automatic resource leveling functionality, and not any other functionality, which may or may not be available to the scheduling tools.
GR002 Tests and test schedules shall be easy and transparent enough to be reproduced with minimal efforts.
GR003 Test schedule as well as leveling results shall be simple enough to be understood and assessed by a human.
GR004 To allow comparison between different tools, test schedule shall use most basic scheduling features, available to most (if not all) scheduling tools, which have automatic resource leveling functionality. This assumes, that some useful resource leveling features may deliberately not be included in testing for the sake of being able to cover a wider variety of tools.
SRSZ001 Schedule size shall not be more than 20 tasks
/* EZ Reasonsing:
*/
SRSZ002 Schedule shall not have more than 7 resources to level
/*?? I took 7 out of my head, so I need an advise on what is the reasonable number?? */
SRGN001 Schedule shall have only Finish to Start links
SRGN002 Schedule shall not have variable resource assignments.
SRGN003 Schedule shall have fixed resource assignments (no skills)
SRGN004 All tasks shall be fixed duration tasks.
SRGN005 Every resource, assigned to work on a task, shall be required to work from the very beginning till the very end of the task.
SRGN006 Tasks shall not be splitable.
SRGN007 Resources, assigned to a task shall have the same percentage of assignment through the whole duration of a task (no curved or time-phased assignments).
SRGN008 Schedule shall have only one calendar, applicable to all tasks and all resources.
/* EZ: this is just to keep it simple. */
SRDR001 Schedule shall be submitted in the format ??TBD??
/* EZ: what format shall we chose not to be biased? One could think of Microsoft Project 2007 or Microsoft Project *.XML, but I personally had problem importing MPS XML file into Primavera. When I googled it, I found out that this seems to be quite often problem.
*/
Printer-friendly version
Send to friend
#Evgeny Z, include Vico Control - resource leveling is one of their core competencies
Evgeny,
If to use Aurora sample jobs, make sure it is something more challenging than the 8 activities job they make reference in their paper, it is too easy for Spider to get absolute minimum for this sample job, finding more than a single prioritization solution that yield minimum was easy.
The following is one example of a solution that yields the minimum, they know but did not showed how to get using their software. Note that you might have to run several times the resource leveling in order to changes in float will yield better results.
http://www.mediafire.com/?6hbmb35ry7krmz5
No rocket science, total float is a common suspect of a prioritization that at times can yield good results, although on real complex problems it will usually be not enough if compared to optimization. If you select DRAG [ascending] you will get the same results, but this is no obvious suspect. Another logical prioritization rule you might try is using resource hours.
Probably they missed how easy it can be with Spider but they are very knowledgeable, their product must be excellent. They know on complex problems even if you use all possible prioritization combinations only algorithms that look for optimization will consistently yield good results.
If you can get a readable display or even better a file of their longer schedule on the same paper this I would like to see.
Also ask about the price for a single deployment. If you have a very special need their customization shall pay off many times, not only for NASA, not only for Boeing but also for Massachusetts General Hospital for which the scheduling of work shifts is very important. To me they are true challenge.
Rafael
Evgeny, resource leveling is not a subject that interests me, but in my travels participating in various industry conferences, I did get the chance to meet Dr. Robert Richards from Stottler Henke Associates, Inc. He developed the resource leveling algorithm for their Aurora scheduling engine. In conversation with him, he drew comparisons for his leveling engine against Spider. It was clear in talking to him that he considered Spider to be his main competition from a gold standard perspective (of being the best).
You might try contacting Rob and asking him for any test schedules where Aurora might have tested better than Spider. This of course, may produce the same sort of bias in having a test schedule that exploits a known special condition that one software has considered and another has not, but it may be the best opportunity for you to find a rival to Spider's leveling engine.
... I see that Stottler Henke is in fact offering:
"Take the Aurora Challenge! and request a free study to benchmark Aurora against your current schedules."
from their product page as linked above.
Vladimir,
I also wish you and all participants of PlanningPlanet happy and productive Y2013.
May be manufactures will not answer, but at least we can agree on fare and reasonable rules of the game. And later anybody (not necessarily manufactures, but may be also enthusiasts) may come up with the Test Schedules.
Regards.
Evgeny
Evgeny,
I don't think that you will get a response.
Test projects that I suggested were created by independent researches in 1995.
If you want smaller example look at sample project in Spider Lite Demo. It uses only two resource units and consists of 10 activities including two milestones.
Best wishes in 2013,
Vladimir
Email: PPadmin@planningplanet.com
Replies