Software for risk assessment

Hello,



Pertmastre is easy to use software. Manageable and competiable with Primaa vera. How ever, You can try Risky Projects software, smart enough to communicate with all major project planning software. Excellent for Risk and Uncertainity forecasting

Hi All,



I have used most types of Risk Analysis tools, and the input data is very important. However with MS Project, I decided just to model Risk not Uncertainty.



Regards

Robyn

Hi

I have used both Crystal Ball and @Risk - they are similar in terms of intent (i.e. add-in to excel to account for uncertainty). Most users will have their preferences which for most users will be about usability rather than functionality. You need to see to decide.

Nozul

Thank you very much indeed for that information.

I have downloaded the two files.

It will make some interesting reading

Thanks again

DS

Hi Guys,



Below is a link to step-by-step activity that I’ve done for our current project.



Schedule Risk Assessment



Thanks



norzul



** just learn about hyperlink yesterday from Bernard Ertl...thanks Bernard. Hope the link works...

Norzul and Ali

Pertmaster interfaces very easily with all Primavera planning software applications, including their Enterprise products (Previously known as P3e and TeamPlay). In fact, in the latest version of Primavera Enterprise (5.0), Primavera have included an evaluation copy (limited to ~25 activities) of Pertmaster (using the name Monte Carlo, previously used for the Primavera product superseded by Pertmaster). Users who wish to upgrade to an unrestricted version have to contact Pertmaster and purchase a licence.

Pertmaster also works "out of the box" with MS Project and Welcom Open Plan. If you want to do Schedule Risk Analysis and Cost Risk Analysis, nothing else comes close to Pertmaster for speed and range of features. Pertmaster’s scheduling engine is written in Assembler, which makes it extremely fast.

For exclusively Cost Risk Analysis, Crystal Ball or @Risk are competitive and a comparison of features and value for money is warranted.



In July, I carried out a schedule risk analysis on the completion of an LNG train in the middle east using Pertmaster, interfacing with P3 3.1. The P50 forecast of 14Nov05 of the Ready for Startup Date (RFSU) I reported in early August came within 1 day of the actual RFSU date (13Nov05). The P90 date was 21Nov05 and the Deterministic Schedule (& Contractual) RFSU date was 31Oct05.

Pertmaster allows you to work with real schedules and make meaningful predictions (but note that it is important to include correlation in your analyses).

I will be reporting on this and another LNG plant schedule risk analysis in an Australian Primavera User Group conference paper next April.



Regards

Colin Cropley

Melbourne, Australia

Dear Ali,



I’m not so sure about P3e, coz what we use is P3 Version 3.0 But please check the http://www.pertmaster.com/primavera/index.htm



Based on the above, I think it’s compatible. Anyway based on your experience, what are the differences between P3e & P3?



Thanks



norzul

has any one used Crystal Ball? Its is supposed to to act as an EXCEL add on. You can check it out at www.decisionengineering.com

I believe this PERTMASTER is very easy to use and its quite user friendly software. It can be used with both P3 and MS Project.

Dear Ali,



Yes recently we used PERTMASTER for the monte carlo analysis for the schedule. Yu can download a trial version from the below website



http://www.pertmaster.com/



Thanks



norzul

[please do not post the same message in multiple threads. Thank you.]

I met Mr. Lev Virine at the PMI Global Congress a few weeks ago. His company is marketing a new program called RiskyProject that uses an Event Chain Methodology for risk management. It looks interesting.



Bernard Ertl

InterPlan Systems - eTaskMaker Project Planning Software

Vladimir

You are right that schedules with short durations invite the use of schedules in scheduling units of hours or even minutes. I haven’t used minutes, but I have done a couple of schedule risk analysis consultancies using hours, one of them where I converted the P3 schedule for completion of an LNG train from days to hours. Many of the completion activities had only a few days of remaining duration. Once the schedule was converted to hours, the minimum remaining duration of one day converted to 10 hours, which was quite workable for analysis, as discussed previously. An interesting problem I encountered in obtaining the schedule risk duration ranges from the construction and commissioning supervisory staff was that they found it difficult to think in hours, as they were used to estimating completions in days. In some cases I chose to assign optimistic durations a few hours earlier than the most likely one provided. Converting to hours also made the analysis results more "discriminating".

Colin Cropley

Colin,

shorter durations invite to measure activity duration in hours or minutes.

I agree that using Monte Carlo for large schedules is not practical.

This is a belated reply to Bernard:

"Colin, practical considerations of the excercise aside, it would seem more natural to me that a more detailed schedule should yield better results as there should be less variance for individual task estimates. Did you have some specific reasons in mind that lead to your recommendation (or was it solely based upon practical considerations)?"



I agree with Bernard that the more detailed the schedule and the shorter the durations & uncertainty ranges, the narrower will be the range of simulation dates or time periods. This can lead to misleadingly narrow results, unless correlation is taken into consideration, to exclude unlikely combinations of duration ranges. Without correlation, each task duration range is taken as being independent of all of the others. For example, to use a tangible example, if there are many activities describing insulation being installed on piping, if the work is proceeding more slowly than planned, it is to be expected that all such insulation activities will be similarly affected, not that their durations will behave perfectly randomly. So we might expect a correlation of +80% or higher (correlation can range from +100% for completely matching duration variation behaviour, through 0% for completely independent activities, to -100% for completely oppositely matching activities, where an increase in the "parent" activity duration leads to a proportional decrease in the duration of the "child" correlated activity.

Correlation is a tricky area in Schedule Risk Analysis, as it is very difficult to be certain that the correct correlation value has been chosen, without substantial statistical data recording the changes in one or more task durations against the changes in another.

So we have to "guestimate".

If anyone has a reliable methodology for selecting correlation values, it would be very helpful to publish it here!



The other reasons for choosing a less detailed schedule are practical:

1) Durations need to be long enough for duration ranges to be meaningful and accurate. If the starting activity is 3 days long, where the scheduling unit is days, the minimum or optimistic duration can only be 2 days (67% of Most Likely duration) or 1 day (33% of ML). If the ML duration is 10 days, the available values for Min/Opt can range down from 90% to 10% in 10% decrements.

2) The Schedule risk analysis will be faster if the schedule has less activities.

Dear Sirs,



There is a Line-of-Balance based schedule planning and control software including a full CPM engine, which has easy-to-use Monte Carlo risk analysis features. This is a very powerful tool because the dependencies of the whole activity (consisting of multiple locations) are copied to all locations. This means that it is possible to model very big projects (10 000 + activities in P3) using only some activities in DYNAProject (50 activities) and using the same schedule logic (all link types with delays supported).



Resource leveling is done automatically in LoB by optimizing work continuity in locations for every trade.



Risk analysis is standard Monte Carlo simulation with pessimistic, expected and optimistic duration evaluations for each location of a task. In addition to showing likelihood of achieving end date and milestones, criticality indexes and other standard stuff, DYNAProject also shows where project flow is likely to be disturbed thus causing extra costs (risk of subcontractor leaving the site and associated come-back delays)





More information available from http://www.dss.fi/pages/index.php?section=English.DSSCompany



..or from www.projectflow.com.au if you are based in Australia, New Zealand or South-East Asia.



Best Regards,



Olli Seppänen

technology manager

Dynamic System Solutions Ltd.

Dear Colin ,



I had gone thru the detailed chain mails on schedule risk analysis ..

I am also involved in project controls , and would like to have some more guidance from you , on actually experiencing the schedule risk analysis . We had done only qualitative analysis .

I am actually migrating to australia , and would like to meet you .



Regards

Bernard,

resource productivity should be normed if you want to be sure that different schedules in your organization are created using the same basis. I strongly oppose the approach when project planner estimates activity duration separately, not using corporate standards, if this activity is typical. Project Office shall create corporate norms that are mandatory for use in any projects that contain typical activities and resource assignments.



When you create project schedule you plan the standard work week, etc. Actual data will show that you had problems if productivity was dropped due to people overwork, etc. Long shifts mean poor planning or poor performance, or risk events. These data are actual. We usually suggest to develop and to use three point estimates for resource productivity and these estimates also serve as the corporate standards.



Other factors:

1) Weather - actual event. It is covered by three point estimates when project schedule is created. Weather is risk factor from the scheduling point of view.

2) Equipment is a resource that is assigned to activity. And of course productivity depends on the set of resources assigned to perform an activity. It is just different type of assignment if different equipment is used. Productivity depends on type of assignment.

3) Type of assignment depends on type of activity, type of resource (resources with different skills are resources of different types) or type of the team (we call it multiresource) that is assigned to perform an activity. Workplace saturation defines activity type.



In any case it is necessary to implement common approach to activity duration and cost estimation within an organization. Productivity norms create this common approach and should be used to be sure that any schedule is based on the same initial information even it is not correct. Project Office reponsibility - to control the norms that are used and improve them if actual data show that initial estimation were wrong.



And of course I mean detailed level - operations.

Do you agree that activity duration depends on assigned resource productivity and if this productivity estimate is wrong then it will lead to the wrong duration estimates for all activities performed by this resource?

Emphasis mine. There are some assumptions there that may not be valid:

• All tasks with common resource were estimated uniformly. This will not necessarily hold true if multiple planners/estimators are generating the plan.
• Productivity is constant. Studies show that productivity drops over time when the working shifts are long and/or there is no time off on weekends. Productivity for a resource 3 weeks into a schedule running 7-12s will be much different than on day 1.
• Activity duration depends on resource productivity. There are many other factors that can impact the actual duration depending upon the nature of the work:
  • Weather
  • Equipment (tool) availability
  • Workplace saturation
  • Fatigue (mentioned previously)
  • Skill level of worker (not all welders are equal for instance)
  • etc.

Variances in estimating can be mitigated to an extent by planning to a high(er) degree of detail.

up to 10,000 activities ..., I would not recommend using such a detailed schedule for schedule risk analysis

Colin, practical considerations of the excercise aside, it would seem more natural to me that a more detailed schedule should yield better results as there should be less variance for individual task estimates. Did you have some specific reasons in mind that lead to your recommendation (or was it solely based upon practical considerations)?



[edit - I see Vladimir posted while I was composing this and addressed this issue more in depth. I share his sentiments.]



Bernard Ertl

InterPlan Systems Inc. - Project Management Software, Project Planning Software

Colin,

risk management is an ongoing activity and in our approach we analyze and simulate risks at least once a week. During this analysis we estimate trends of probabilities to meet project objectives successfully. Performance is measured and managed on the detailed level and there are problems with creating and using less detailed model. One of them you had mentioned - activity dependencies are defined on the detailed level and less detailed model will not be accurate. Besides the level of detalization is defined by resource levelling requirements - you will not be able to level resources if your schedule is not accurate about resource requirements. And you shall simulate risk events along with uncertainties, so three point estimates are not sufficient. I will download Pertmaster software (I had known their scheduling package in the beginning of 90-s but did not try their risk simulation software) but in any case my concerns refer not to the software but to the approach. Proper application of Monte Carlo technique is too time consuming to be practical. Inproper application of this technique will produce unreliable results.



The computer model of the Caspian pipeline (only 750 km portion) construction consisted of near 10000 activities, 1145 resource types (not including materials - there were near one hundred of them in the model). The number of activities was defined by resource levelling requirements and could not be lowered. Initial information included resource productivities and uncertainties were linked with their estimates (too hard to input correlations manually), there were risk events, like flooding, problems with obtaining some permissions, etc. Some of these risk events may lead to the productivity problems, others - to the scope changes (conditional branches). It is just an example illustrating my concern about reliable applicability of the Monte Carlo technique to the real projects. In Spider Project resource levelling of this project takes near 3 minutes and it is only one pass. I don’t think that other software can produce resource constrained schedule of the project of that size faster.

I don’t think that 200 iterations will be sufficient for this project. I am not worried with the possible error (initial data are not that reliable), I want to be sure that the next calculation will produce the same result. As I wrote earlier risk level is not that important as the trend of that level during project performance. Trends produce early warning and are very good indicators of the project performance. You cannot evaluate trends if the calculation results are not stable.

There is one other issue. Resource levelling algorithms used by different software are different too. If you use Pertmaster for risk simulation and Primavera for project management then you may have a problem because different schedules were simulated. What do you think about this?

Vladimir

Vladimir

Although Pertmaster is very fast and can standardly cope with up to 10,000 activities (more if a special version is obtained from Pertmaster), I would not recommend using such a detailed schedule for schedule risk analysis. You need a level 2 or perhaps 3 schedule, which has task durations of at least 5-10 days, to enable realistic duration ranges to be defined for the tasks. Normally 3 point duration estimates are made for each task: Minimum/ Optimistic/ P10; Most Likely/ P50 (normally assumed = Original Duration); Maximum/ Pessimistic. If a task original duration is only 1 or 2 days, there is no scope to define an accurate Minimum/ Optimistic/ P10 duration.

On the other hand it is important to include the significant dependencies in the schedule, which are normally at a fairly detailed schedule level.

So the detailed project schedule often has to be summarised before schedule analysis can be performed. This usually represents the largest portion of time and effort in carrying out effective schedule risk analysis. A good analogy is painting a house - the better the surface preparation, the longer the paint will last after it is applied.A few months agao, I did a schedule risk analysis for an offshore gas field development project. The assignment took nearly 3 weeks and all but the last few days were taken up in preparing a suitable summary project schedule for importing from Primavera into Pertmaster.

Normally a schedule risk analysis converges within about 200 iterations (the mean and variance of the probability density distribution cease to change by more than 1% from one iteration to the next).

I suggest you download the free Pertmaster evaluation software and try it. It is about 28MB, so a broadband Internet connection makes a difference, but it is still possible with a dialup connection

Colin Cropley

Colin,

thank you for your reply.

But I am not sure that 5000 repetitions is sufficient for projects consisting of 10000 or more activities. This size of projects is quite usual in construction. I suspect that using this number of repetitions you will receive different estimates each time you run Monte Carlo simulation.

If you level resources in projects of that size then each scheduling try will take minutes. Even if you restrict the number of repetitions by 5000 (I repeat that this number does not look sufficient) then Monte Carlo simulation will take hundreds of hours. But you need to do it not only during planning stage but also during project performance. This is my concern about the number of necessary iterations and stability of simulation results.

The number of project risk events (and corresponding conditional branches) that should be simulated may be large too - and it means that project model may become several times larger than the initial and will become too complicated to work with on a day-to-day basis.

Do you agree with my concerns or you have another experience and oppinion?

Vladimir, Responding to your questions in turn:

1. A project director with many years experience with Bechtel told me that they always ran at least 5,000 Monte Carlo simulations in an analysis. Generally, the key parameters (mean, standard deviation etc) of the probability distribution doesn’t change much above 1,000 iterations, but the smoothness of the histogram of "hits" per time or cost interval does.

2. I agree that activity duration depends on the productivity of the assigned resources and that activities related by shared resources are almost certain to have some level of correlation. I don’t know about other applications but Pertmaster is certainly capable of having correlations applied to groups of activities between -100% to +100%. Assignment of correct correlations is crucial to obtaining reliable probability analyses. An attempt to quantify the effect of correlation on probability analysis is provided in a paper "Scheduling with Uncertain Durations". A link to this paper was provided by Bernard Ertl, the moderator of this forum in the posting 3rd from the bottom of this thread (dated 29 Nov 2003).

You could do a Monte Carlo analysis on the productivity factors using @Risk, instead of using a schedule risk analyser like Pertmaster. However, I like the flexibility that the schedule risk analysis provides, particularly if there could be tasks that may or may not occur (task existence uncertainty or probabilistic branching. And tasks have uncertain durations for reasons other than resource productivity variation.

3. Your impression that Monte Carlo analyses are performed without the reliability of the simulation results being assessed is no doubt true. By definition Monte Carlo analyses are uncertain. And they depend critically on the reliability of the source data for duration ranges and task existence probabilities.

However, they should provide more information on which to make project decisions than a single, deterministic schedule duration and project cost.

Colin Cropley

What do you think about the number of tries you need to perform using Monte Carlo simulation for large projects?

Do you agree that activity duration depends on assigned resource productivity and if this productivity estimate is wrong then it will lead to the wrong duration estimates for all activities performed by this resource? Activitiy durations are correlated and I don’t know the Monte Carlo simulation software that takes these correlations into account. In this case the results of Monte Carlo simulation are wrong.

There are many other issues that also deserve to be discussed.

I have an impresiion that people use this approach without estimating the reliability of the Monte Carlo simulation results.

This is a specific, second response to Paul’s message below regarding risk analysis and assessment for projects prone to natural calamities like earthquakes, floods, etc. Paul cites the example of a cross-country oil/gas pipeline.

I have been involved in feasibility studies on an international gas pipeline from Iran to Pakistan and India on and off since 1996. The pipeline route crosses one of the most earthquake-prone regions of the world, as the terrible earthquake centred on Bam in January illustrated, as well as areas subject to monsoonal flooding.

We used the Risk Management process set out in Australian / New Zealand Standard AS/NZS 4360:1999 Risk Management. This is similar to the PMI "Guide to the Project Management Body of Knowledge" (PMBoK 2000) Chapter 11 Risk Management process. It includes Risk Identification, Qualitative and then Quantitative Analysis, Risk Response planning to manage the most threatening risks to the project, then Risk Monitoring and Control through the project. I would encourage you to request a free 38 page "White Paper" (pdf) available on the Pertmaster website (www.pertmaster.com) titled "Project Risk Management Procedures",by Bob Harvey of Canadian Risk Technologies Inc. The first 16 pages describe well the generic risk management processes outlined above that we followed.

In simple terms, apart from the risk of terrorism (these days unfortunately a very significant risk, particularly in that region), the other risks to the project came down to cost allowances, except for the monsoonal flooding.

The monsoonal flooding had two effects:

1. A construction calendar excluding the flooding months was used for pipe laying in the flood plain region.

2. A cost allowance was made for concrete casing of the pipe to overcome buoyancy where there was a risk of the pieline literally floating to the surface.

For crossing known earthquake fault lines, cost allowances were made for special reinforcing of the pipeline (heavier wall thickness) and special sloping trenching to allow the pipe to spring out of the ground in an earthquake and avoid rupturing.

It is a surprising fact that pipelines are not commonly ruptured by earthquakes because even large diameter pipelines are quite flexible.

Colin Cropley

I strongly recommend that readers of this forum take a look at a Monte Carlo type risk analysis application produced by Pertmaster (www.pertmaster.com), which I first reviewed for the Victorian Primavera User Group in Australia two years ago. Pertmaster is a UK company that started over 12 years ago selling a project planning and scheduling application. In the last 5 years they have concentrated on schedule risk analysis. The application is very fast and markedly superior to Primavera’s Monte Carlo product, so much so that Primavera has made Pertmaster a Primavera Partner and is actively helping Pertmaster to promote Pertmaster use by Primavera’s own client base.

Pertmaster will read and write to P3 files (& SureTrak) as well as P3e/TeamPlay. It also reads and writes to MS Project. It will perform time and cost risk analysis, including cost analysis independent of time. It includes probabilistic tasks and branching, as well as positive and negative correlation and tornado diagrams to identify schedule and cost drivers.

This is Not a product promo /spam, as I was a member of Planning Planet long before trying Pertmaster. However, I did suggest that this Schedule Risk forum be started by Planning Planet after I became a Pertmaster Solutions Partner (provide consulting and training in Pertmaster) and was asked by a Pertmaster user whether a forum could be established for Pertmaster users. I felt that a generic Schedule Risk Forum on the Planning Planet would be more useful, and would give users of other software, such as @Risk for Project, Monte Carlo and Open Plan’s Monte Carlo functionality, as well as others with a developing interest in the subject, the opportunity to participate in helpful discussions.

So far, the forum has fulfilled those hopes!

Colin Cropley

Risks like flood shall be simulated, eartquakes and other force majeures shall be insured. We have experience of planning risks in pipeline construction projects (Caspian pipeline construction, Jamal - Europe pipeline and others)using other software. So let’s hope that somebody will explain how to use Monte Carlo for Primavera P3 to simulate not only uncertainties but risk events too. I am rather sceptical about applicability of Monte Carlo simulation to the large projects.

Regards,

Vladimir

Good point Guy.



I recall a discussion where material procurements were being scheduling at the latest possible moment. This, of course, invites a measure of risk that the material will not be delivered on time for any number of reasons (out of stock / back-ordered, damaged merchandise needing replacement, force majuer, Murphy’s Law, etc.). I do not think that this type of risk can be accurately qualified/quantified/modeled by variable duration estimates.



Bernard Ertl

InterPlan Systems Inc. - Project Management Software, Project Planning Software

Some interesting debate I came across recently concerned modelling Risk and Uncertainty. With a three point estimate for schedule risk analysis we are really only modelling Uncertainty in activity definition, as distinct from Risk.



Is this a good definition or not? I came this idea recently and feel it is worth sharing and seeing what other views are out there about risk schedule analysis.

Software Monte Carlo cannot provide quantitative analysis. The study of probability only.



RisGen is to identify a risk, study the impact, any remendy, cost involve for pre-castion or action, etc.

The following excerpts are from a very nice technical resource describing various project management calculations:

• PERT - While the PERT method has been made widely available, it suffers from three major problems:
  1. The procedure focuses upon a single critical path, when many paths might become critical due to random fluctuations. For example, suppose that the critical path with longest expected time happened to be completed early. Unfortunately, this does not necessarily mean that the project is completed early since another path or sequence of activities might take longer. Similarly, a longer than expected duration for an activity not on the critical path might result in that activity suddenly becoming critical. As a result of the focus on only a single path, the PERT method typically underestimates the actual project duration.
  2. It is incorrect to assume that most construction activity durations are independent random variables. In practice, durations are correlated with one another. For example, if problems are encountered in the delivery of concrete for a project, this problem is likely to influence the expected duration of numerous activities involving concrete pours on a project. Positive correlations of this type between activity durations imply that the PERT method underestimates the variance of the critical path and thereby produces over-optimistic expectations of the probability of meeting a particular project completion deadline.
  3. The PERT method requires three duration estimates for each activity rather than the single estimate developed for critical path scheduling. Thus, the difficulty and labor of estimating activity characteristics is multiplied threefold.
• Monte Carlo - This technique calculates sets of artificial (but realistic) activity duration times and then applies a deterministic scheduling procedure to each set of durations. Numerous calculations are required in this process since simulated activity durations must be calculated and the scheduling procedure applied many times. For realistic project networks, 40 to 1,000 separate sets of activity durations might be used in a single scheduling simulation. ... The disadvantage of Monte Carlo simulation results from the additional information about activity durations that is required and the computational effort involved in numerous scheduling applications for each set of simulated durations. For each activity, the distribution of possible durations as well as the parameters of this distribution must be specified. For example, durations might be assumed or estimated to be uniformly distributed between a lower and upper value. In addition, correlations between activity durations should be specified. For example, if two activities involve assembling forms in different locations and at different times for a project, then the time required for each activity is likely to be closely related. If the forms pose some problems, then assembling them on both occasions might take longer than expected. This is an example of a positive correlation in activity times. In application, such correlations are commonly ignored, leading to errors in results. As a final problem and discouragement, easy to use software systems for Monte Carlo simulation of project schedules are not generally available. This is particularly the case when correlations between activity durations are desired.

Scheduling with Uncertain Durations



Bernard Ertl

InterPlan Systems Inc. - Project Management Software, Project Planning Software

Doesn’t Monte Carlo analysis give quantitative results?

Can anyone describe Risgen and what it does please?