Guild of Project Controls: Compendium | Roles | Assessment | Certifications | Membership

Tips on using this forum..

(1) Explain your problem, don't simply post "This isn't working". What were you doing when you faced the problem? What have you tried to resolve - did you look for a solution using "Search" ? Has it happened just once or several times?

(2) It's also good to get feedback when a solution is found, return to the original post to explain how it was resolved so that more people can also use the results.

Estimating the expected lost days to weather.

26 replies [Last post]
Rafael Davila
User offline. Last seen 4 hours 33 min ago. Offline
Joined: 1 Mar 2004
Posts: 5229

In order to model the weather effects basically two things are needed, the software capable of doing so and the input data. While I have access to computer models that can theoretically do it the problem I am facing is with the data to be used.

We can get weather data for a period of over 30 years in most localities but how this translates into the format that is to be imputed into our models?

Recently the trend of government agencies is toward requiring in a very vague language for contractors to consider weather into their models and take 100% of the risk on what is expected and only what is way above normal (whatever it means) that had an impact is to be considered a shared risk. Although I believe the vague language will not hold in court I am looking for some guidance on what is an acceptable methodology for this purposes. I get completely lost when the agencies talk about NOAA and at the same time they provide their own estimate, I do not know which one prevails.

As I am from a tropical region where it rains frequently I would like the procedure to be done using NOAA and the NWS (National Weather Service) data for Dorado Puerto Rico to see if we can agree what are the average expected days to be lost for every month and expand to a probabilistic distribution to be used in Monte Carlo models.

 photo doradorain_zps5fe173c2.png

Thanks in advance for your contributions,

Rafael

Replies

Rafael Davila
User offline. Last seen 4 hours 33 min ago. Offline
Joined: 1 Mar 2004
Posts: 5229

The following link provides the file where I estimated the Mean, P10 and P90 values.

http://www.mediafire.com/view/?4wjh53i3hv1179x

 photo MEAN-P10-P90_zps220a743f.png

Best Regards,

Rafel

Rafael Davila
User offline. Last seen 4 hours 33 min ago. Offline
Joined: 1 Mar 2004
Posts: 5229

Dennis,

Tropical waves are more frequent than hurricanes, at times with more rain than a hurricane and can have a substantial residual effect after the rain event already included in NOAA records, might be worth investigating.

Now that I know how to estimate the basic rain calendar I will look for the tropical waves calendar. NOAA has a lot of data available, it should be there, but understanding the data might take me more time. I am trying to get in contact with an ex-coworker who specializes in hydrology but have not being able to find him.

We get tropical waves every other week on peak season and at times they collide, at times they come one after the other, some coming from the west generated by cold fronts from the US and some coming from the east generated in Africa.

In some jobs weak waves make and impact in other jobs not even the strongest have a residual impact other than the rain event in NOAA records. Probably I would look for the monthly distribution of tropical waves, depressions and storms that never made it into a Hurricane but that poured over certain amount of rain, say over 3 inches. This I will use to trigger a sequence of weather event that will represent 2 extra days lost after the rain event. I have no idea how it will come out.

I believe I will end up using about 3 weather calendars, one for rain using rainfall >=0.1 in/day, another for rain using rainfall >=0.3 in/day and another for the tropical waves, depressions and storms . These will be applied differently to different jobs, to different activities as the need be.

Best regards,

Rafael

Dennis Hanks
User offline. Last seen 8 years 1 week ago. Offline
Joined: 17 Apr 2007
Posts: 310

Rafael:

It has been fun and thanks to Stewart at NOAA I now have the dp01 report which makes this process a lot easier (assuming a rain day is >.1"/day).  As to hurricanes, my analysis for Lake Charles, LA suggests that too much variability exists to make reasonable assumptions as to impact.  It is almost impossible to predict when a hurricane will occur, how strong it will be, and therefor how much damage it might inflict.  For Puerto Rico, it may well be worth the effort.  I suggest [http://csc.noaa.gov/hurricanes/#] it graphically represents the extreme variability of hurricanes as a starting point - is it a significant event?

1538
sjia004.jpg

You may want to look at my attempt to model a hurricane for Lake Charles. http://www.youtube.com/watch?v=kphHbqMu7LY&feature=youtu.be

Dennis

Rafael Davila
User offline. Last seen 4 hours 33 min ago. Offline
Joined: 1 Mar 2004
Posts: 5229

Raymund:

RE: Are the lost days to weather when accumulated will be granted as Extension of Time?

  • Days lost to rain are usually granted when it rains above normal, one of the issues is what is normal, another what is the rainfall amount threshold that will trigger the impact. Simple contract rules that make it clear the risk are good but rules subject to interpretation are not good for purpose of contracting rules. I am looking for the rules that shall be subject to interpretation of the person that bears the responsibility for the delivery of the job.

RE: Clients will likely consider to grant those lost days because it is a natural cause beyonds contactor's nor client's control.

  • Below the established amount it is 100% risk to the contractor after the amount it is a shared risk, time extension is granted no liquidated damages or extended overhead are applied.
  • In some (few) jobs it make sense to shift all risk to the Contractor but usually in such cases the rules are clear.
  • In some cases (many) not sharing the risk can backfire, contractors will work under adverse conditions yielding poor quality of work. If I bear all the risk for rain, I will pour the slab no matter how dark the clouds are, if it comes out bad, the owner will have to accept a repair that is not as good.

RE: Risks are expected from unexpected sources. There are a lot of Risk we expect to happen but sometimes before it is about to happens, preventive measures were introduced. Sometimes Risks had come with unexpected low impact.

  • Agree.

RE: With regard to your researches to estimates the lost days to weather, is the reason to determine it earlier would means the impact will be considered in the mark up for Pricing of Tender?

  • Impact is to be considered as a possible cost in the pricing, you never know costs ahead it is always an estimate. In addition the main idea is to make good planning with the use of appropriate buffers calculated in a rational way.

Best regards,

Rafael

Raymund de Laza
User offline. Last seen 30 weeks 4 days ago. Offline
Joined: 23 Nov 2009
Posts: 762

Rafael,

Are the lost days to weather when accumulated will be granted as Extension of Time?

Clients will likely consider to grant those lost days because it is a natural cause beyonds conrtractor's nor client's control.

Risks are expected from unexpected sources. There are a lot of Risk we expect to happen but sometimes before it is about to happens, preventive measures were introduced. Sometimes Risks had come with unexpected low impact.

With regard to your researches to estimates the lost days to weather, is the reason to determine it earlier would means the impact will be considered in the mark up for Pricing of Tender?

 

 

Regards,

 

Raymund

Rafael Davila
User offline. Last seen 4 hours 33 min ago. Offline
Joined: 1 Mar 2004
Posts: 5229

Dennis,

I thank you a lot because it is the first time I looked at NOAA for the data, at the beginning I did not know where to look and the exercise have been a positive learning experience.  To me the question now becomes more an issue of how to define an expected number of days to be lost to rain, it is relatively easy to create the weather calendar after you know from where to pull down the data.

I liked your use of bins to have a better picture of the distribution curve; I did not use it because it is easier to create the chart without the use of bins, a small saving.

The fact that nobody joined us makes me believe that the requirement to look for weather data is overlooked by the great majority of schedulers.

Because the Monte Carlo functionality in Spider is new I still got to learn how to use it. After I looked for some references I found Primavera Risk methodology in agreement with I believe is correct modeling with the probable exception on what distribution curve they are suggesting and making available. It is not that I knew about the difference between normal and log-normal distribution curves, I learned about it yesterday. As I told you I am not a mathematician and do not believe many schedulers at home are mathematicians, most probably a project engineer required to do it  as part of his many other functions.

Even when the available data only provides 50 points per monthly histogram it shows from year to year there can be significant variation in the number of days with rainfall above any given threshold. I do not believe this is enough to consider it statistically enough for considering valid for purposes of contractual use.

I believe in the value the models provide even in the lack of precision. In the same way I believe in the models of NOAA when tracking a hurricane, every year they improve on the quality of early warning saving lives and money. Here they are heroes, on hurricane season they bring the hurricane hunters, provide us with much needed information the satellites nor the buoys can provide.

I would like your comments about how to model the effect of other weather events like hurricanes and the frequent storms that do not make it a hurricane but that can bring more flooding than some hurricanes. I believe the initial rain effect is considered under the NOA statistics so it should be a matter of using a different methodology for this purposes. A random trigger that will have an impact after the initial rain impact, of several days or even weeks in case of a hurricane. Although considered as "acts of god" in our contracts they have their impact and are of interest.

I would dare to say impact of events such as flooding or consecutive rain days can be localized. In most of my jobs the effect of flooding have been none with the exception of a river water intake for a potable water plant and some works on an irrigation canal. For these jobs impact of these events are frequent and must be considered. Again I believe it shall be the sole responsibility of the Contractor to make the assumptions and bear the responsibility of his own planning.

Again many thanks for your assistance.

Rafael.

Rafael Davila
User offline. Last seen 4 hours 33 min ago. Offline
Joined: 1 Mar 2004
Posts: 5229

Dennis,

The infatuation of many with the "normal distribution" can be the source of many errors. Usually activity duration distributions are not symmetrical as well as costs and perhaps even rain. I would ban the use of normal distribution in CPM practice as to avoid such missuses of the normal distribution, in most cases even a triangular distribution would be a better fit.

http://stat.ethz.ch/~stahel/lognormal/bioscience.pdf

Life is not always symmetric, we have better vision on one eye, we have a longer leg and arm and perhaps a testicle, maybe funny but that is how it is. We got to stop assuming everything fits on a symmetrical model as if tossing a coin.

For how the distributions looks for simplicity I would use a log-normal distribution with minimum and maximum values and not infinite values as is the case of a normal distribution. The cumulative distribution shall be a better fit as well as P10 and P90 values.

Please contact me via PP and I will send you all the references, do not publish your personal e-mail, better create a temporary Gmail account.

Best Regards,

Rafael

PS. I am not a mathematician, believe me I am struggling with this and some of my statements might be in error. I am an old grumpy civil engineer fighting with the numbers.

Dennis Hanks
User offline. Last seen 8 years 1 week ago. Offline
Joined: 17 Apr 2007
Posts: 310

Rafael:

Still having difficulty downloading the file.  Joined Mediashare and still nothing.  Thought it might be Google Chrome, but when I tried IE I was unable to log into Mediashare.  Awaiting a lost passwork email from Mediashare.

I was unable to log back into Mediashare.  Stewart at NOAA was very helpful and the dp01 file makes this an almost trivial exercise.  See results below.  I have uploaded my files to Mediashare, I will send the URL when I can log back in.  There has to be a better way.  I for one resent having to use iLivid to download.

 

1535
sjia001.jpg
1536
sjia002.jpg
Trends are not uniform/consistent, but interesting. [other QTs omitted to save space]

1537
sjia003.jpg

You might be right about 'normalcy', but if you compare the January histograms (30 v. 55) the most egregrious, there seems to be a trend towards a bell curve.

Rafael Davila
User offline. Last seen 4 hours 33 min ago. Offline
Joined: 1 Mar 2004
Posts: 5229

Dennis,

Try the following links and let me know, I will edit prior reference with the one that works for you so no one have problems, the idea is to have as many people as possible to double check. It is different to the one I posted and wonder why I got different links.

http://www.mediafire.com/view/?62f8aoz41iqbttq

alternate link is

http://www.4shared.com/office/A7QPC5_K/NOAA_SJ_Rain_Data.html

for a few more years (1955 to 2011) and distribution displayed as % of Grand Total. Fifty years of rain history is not much as it will yield only 50 points for each of your 12 monthly requency distributions.

http://www.4shared.com/office/DX0TGiWv/NOAA_SJ_Rain_Data_1955-2011.html

You are right, the data can be summarized as you said but I wanted to keep other data as to be able to look at the data for other thresholds I might get by interpolation as well as to define temperature calendars for hot weather concreting. Also if you can see the tables in a similar way they are at NOAA it is easier to follow what I did.

Once I got a view on each table I selected all and then Copy and Paste into my worksheet. I cleaned up the extra lines and all took me about 20 minutes. The idea is you can do it in similar amount of time for your locations. If we can learn to pull down the data from NOAA in a reasonable amount of time for any location we all win.

About how to get the distributions using pivot tables the following links might help:

http://datapigtechnologies.com/blog/index.php/creating-a-frequency-distribution-with-a-pivot-table/

http://support.microsoft.com/kb/214269?wa=wsignin1.0

For a 10 point distribution template I prepared some time ago:

http://www.4shared.com/file/ZC0Ajpma/Probability_Distributions_usin.html

The following reference might help to understand how the two ways of defining inclement weather can be applied.

  • Event with results in an uncertain number of non working days scattered through a period, e.g. rain or snow.
  • Event with results in a single block of non working time with a probability of occurrence, e.g. chance of a hurricane in a period.

http://www.slideshare.net/p2rahi/schedule-risk-analysis-sra-by-pedram-daneshmand-14jan2011

At the moment we are concentrated on the first definition and we are looking for the data as shown in slide 22, Risk Register Rain Details.

 photo rainregister_zpsacbade66.png

Note that it is a distribution what is of concern and not mere averages, any procedure that pretend to model rain using deterministic values or averages is flawed.

Best Regards,

Rafael

Dennis Hanks
User offline. Last seen 8 years 1 week ago. Offline
Joined: 17 Apr 2007
Posts: 310

Rafael:

Having difficulty downloading your file.  I would have expected to see the data in tabular form [12x30 matrix - since you did only 30 years] with a mean and standard deviation of number of rain days (>.1") per month.  Following some of the links while viewing your data, I saw an interesting file - DP01 -for SJIA, but was unable to add it to my cart.  If these files (DP01) exist for other areas, my problems may be over.  Currently, I am having problems with pivot table filters.  What once worked, is no longer.  

I think we will have to set up direct email file transfer.  In the meantime, I am going to try to get that DP01 file.

Rafael Davila
User offline. Last seen 4 hours 33 min ago. Offline
Joined: 1 Mar 2004
Posts: 5229

The following Charts are the frquency distribution for number of days with rainfall above 0.11 in for the last 30 years at San Juan Int. Airport.

(Note: some values are missing for some months of 1993 in the NOAA tables)

This is what it is and do not believe can be normalized.

 photo RAINFALLDISTRIBUTION_zps99721c00.png

Based on NOAA Data fro 2010 we had 142 days with rainfall over 0.10 in. Using this as the threshold and neglecting cummulative effect we worked 50% of the year and 50% partying.

 photo 2010rain_zps755548a9.png

Feel free to download the worksheet with the data and a direct link to the NOAA Tables.

http://www.mediafire.com/?64u0nubb1d8df

Stephen Devaux
User offline. Last seen 17 weeks 6 hours ago. Offline
Joined: 23 Mar 2005
Posts: 667

"BTW: PERT was setup because back then there was insufficient computational power to run mc simulations on project schedules so the pert equation [(min+4likely+max)/6 = expected duration was developed. It was better than nothing and did try to account for uncertainty."

 

Funny, that's how I feel about Monte Carlo! (Provided the output -- almost always based on very inadequate input in terms of both estimates and distribution shapes -- does not become a self-fulfilling prophecy, which it often does. Garbage in, Gospel out.)

Fraternally in project management,

Steve the Bajan

Rafael Davila
User offline. Last seen 4 hours 33 min ago. Offline
Joined: 1 Mar 2004
Posts: 5229

Dennis,

RE: I do not understand this statement: Averages are not good enough for a Monte Carlo Run, these averages are entered as deterministic values. Similar to activity duration where you enter a distribution and not a single average number. This is a the classical error of original PERT that was clarified long ago.

RE: The report you cite seems to have monthly averages - likely value, and what appears to be the standard deviation - which times 1.28 would be the P10 (min) and P90 (min) values. Once adjusting for make-up days you should be able to contruct an OPRA weather calendar. The only problem is you are limited to the thresholds of .01, .1, and .5 which may not be appropriate, but somebody has done the work. You say that it is relatively simple to set up the appropriate query, let's hope so.

  • The reports I cite provide measured monthly data not averages, in any case they compare the data to some average for which I do not know the distribution spread, needles to say I do not know the shape of the distribution. A distribution is needed for Monte Carlo run a few schedules with months having more than the average and also month with less than the average. It is well known the results are not necessarily the same as if you use averages only, the way critical path works such flawed procedure usually yields unreal and always optimistic results.
  • Depart from normal can be 1, 2 or any other standard deviations apart, what the tables display are depart from normal and not necessarily the value for 1 standard deviation, in occasions can be 1 deviation but not always.
  • I expect the distribution for days lost to rain be shifted to the right, not a normal distribution.
  •  photo distributionshapes_zps2e68b718.png

RE: BTW: PERT was setup because back then there was insufficient computational power to run mc simulations on project schedules so the pert equation [(min+4likely+max)/6 = expected duration was developed. It was better than nothing and did try to account for uncertainty.

  • Original Pert, before Monte Carlo was in error, it was immediately discredited and abandoned. Monte Carlo general use came in the 80s some 30 years after it was disclosed that the original PERT Method was flawed. Monte Carlo came into general use as soon as the PC arrived, it was previously accessible to those with access to corporate mainframe computers.

I got an excellent response from NOAA today the same day I asked for assistance. Some of our institutions stand above the average.

  • Another option very useful in terms of engineering is precipitation frequency data; this is a direct link:http://hdsc.nws.noaa.gov/hdsc/pfds/pfds_map_pr.html If you want to do it through our web page, follow these steps: go to our main page: http://www.srh.noaa.gov/sju/, Click on the climate tab ,then on local data/records, and select Precipitation Frequency Puerto Rico.
  •  photo noaaraindata_zps35ed85a2.png
Dennis Hanks
User offline. Last seen 8 years 1 week ago. Offline
Joined: 17 Apr 2007
Posts: 310

Rafael:

I do not understand this statement:  Averages are not good enough for a Monte Carlo Run, these averages are entered as deterministic values. Similar to activity duration where you enter a distribution and not a single average number. This is a the classical error of original PERT that was clarified long ago.

The report you cite seems to have monthly averages - likely value, and what appears to be the standard deviation - which times 1.28 would be the P10 (min) and P90 (min) values.  Once adjusting for make-up days you should be able to contruct an OPRA weather calendar.  The only problem is you are limited to the thresholds of .01, .1, and .5 which may not be appropriate, but somebody has done the work.  You say that it is relatively simple to set up the appropriate query, let's hope so.

BTW: PERT was setup because back then there was insufficient computational power to run mc simulations on project schedules so the pert equation [(min+4likely+max)/6 = expected duration was developed.  It was better than nothing and did try to account for uncertainty.

Rafael Davila
User offline. Last seen 4 hours 33 min ago. Offline
Joined: 1 Mar 2004
Posts: 5229

For the record, as I find this parallel discussion relevant I am copying the following post.

David,

RE: My preliminary analysis for SJIA suggests that it will be more on the order of 8 days per month (8.41) with a range from 4-14 days, but that is only one year (1967) and this would be further reduced if there was a make-up Saturday available, say 4 days per month, so you could expect to lose 4-5 days per month to rain if you were working near the San Juan International Airport. My limited experience to date with the variability of weather data would be to question any analysis done on only one year. Is 4 days per month closer to actual experience? What is actual experience?

  • Days lost to rain is a function of the threshold used, which one are you using to predict it is a day lost to rain 0.10 in, how many days on a single month report rain above 0.10 in?
  • Keep it simple lets talk abut calendar days as if all represent workdays.
  • Can you post the data so I can take a look at it.
  • Can you disclose your computations?
  • Averages are not good enough for a Monte Carlo Run, these averages are entered as deterministic values. Similar to activity duration where you enter a distribution and not a single average number. This is a the classical error of original PERT that was clarified long ago.
  • Monte Carlo shall model some rainy months (of course with rain occurring on different dates) as well as some dry year months not the average month with the same amount of rain days but on different dates. For some runs the extra amount of days can render near critical activities critical, these must be considered otherwise the model will be flawed.

If you have during a month a single activity that is impacted by a rain calendar changing the dates but not the amount of rain dais is not going to make any difference, is going to be dead wrong.

Of course first things first checking on the average before going into the distribution is not a bad idea., what you are getting for the average seems reasonable for a single month of a single year. Yes at times this will happen, at time will be no rain, at times will be more.

Wow 7 days is a full week, was it a hurricane month? Seems like a close encounter but not a direct hit, still too much in the absence of a hurricane close encounter. It seems to me like the numbers NOAA yields for a mere 1/8 in of rainfall, by our standards almost no rain at all if on a single day.

We have no issues with NOAA other than how difficult it is to get the data, we have issues with the threshold that yield way too many days expected to impact the job making it impossible for the contractor to be able to claim rain days on months out of hurricane season.

The following are NOAA numbers, days with rain bellow .20 or .30 here do not make it enough to stop site works. Seems like you are using 0.10 in as the threshold and it is way too little rain.

 photo PRData_zps4002a77a.png

Can we move any further discussion to the Risk Forum where we are having a parallel discussion on the theme? I find your questions of most interest and do not want to miss a single one, in addition I want to avoid keeping out of this thread central topic.

Best Regards,

Rafael

Rafael Davila
User offline. Last seen 4 hours 33 min ago. Offline
Joined: 1 Mar 2004
Posts: 5229

Dennis,

RE: IMO it should be the responsibility of the contractors to develop these 'weather calendars', they are the ones with the data and the exposure. If I was a dues paying member of AGC, CII, or other contractor organizations, I would expect them to provide this data. Why would you expect the owners to perform these studies? That aside, and assuming that "Depart from normal" is the monthly standard deviation, and the .1" is the appropriate target, and you have the other monthly data, then you have all you need to construct your OPRA Weather Calendar. If any of those assumptions are not correct then you will have to do it yourself. Also, assumes that we are using current data.

  • By not being clear as to the threshold they make it impossible to know what is the Contractor Risk, if they specify 0.10 in as the threshold they are making it as if 30% of the time is expected to be a day lost to rain with the effect that all risk  is for practical purposes kept on the Contractor unless a Hurricane occurs.
  • I do not recall a direct hurricane hit for August of 2012 but the rain data says 13 days with rainfall above 0.10 in close to 50% of the whole month lost to rain, such claim would be ridiculous.
  •  photo august2012_zps13e6b92f.png
  • Maybe the AGC provide the data but the contracts call for the use of NOAA Data. If I were an AGC member I would suggest no single day are expected to be lost to rain shifting all risk away from the Contractor.

RE: NOAA provides the raw data at no cost, which is about what I would expect. Since individual needs will vary - what rate constitutes a rain day, a snow day, a hot day, a cold day, etc. - I can see where this might be onerous. That said, the Australian Board of Meterology seems to be providing this data. If such a query could be constructed that could extract this data, it is beyond my capabilities. I would gladly work with anyone who might be able to do the proper programming that would incorporate various user specific filters - assumes Access using CSV data.

  • Such query could be constructed easily by using easy to use database software. It is a matter of providing the tables in a usable format that can be easily consolidated and queried.

RE: Does your company collect rain day data - actual days lost to rain on its various projects? If they do, it will be useful when trying to validate the models. Whatever we construct may not have any predictive value if it cannot be properly validated.

  • I work on my own but will take a look at some schedule updates of my clients, the information is there if I still have access to the files on old P3 and SureTrak format. What you propose makes 100% sense to me although it will be from another regions the data I might have. I have a Guaynabo Job, where it rains twice as much, it is at the center of our rain corridor.

For purpose of getting the methodology we can waive the issue on fairness for the moment, we can assume all risk is on the Contractor and agree on whatever threshold. Latter on once we agree on the methodology then we can go back to the issue on the threshold selection.

Best regards,

Rafael

Dennis Hanks
User offline. Last seen 8 years 1 week ago. Offline
Joined: 17 Apr 2007
Posts: 310

Rafael:

IMO it should be the responsibility of the contractors to develop these 'weather calendars', they are the ones with the data and the exposure.  If I was a dues paying member of AGC, CII, or other contractor organizations, I would expect them to provide this data.  Why would you expect the owners to perform these studies?  That aside, and assuming that "Depart from normal" is the monthly standard deviation, and the .1" is the appropriate target, and you have the other monthly data, then you have all you need to construct your OPRA Weather Calendar.  If any of those assumptions are not correct then you will have to do it yourself.  Also, assumes that we are using current data.

NOAA provides the raw data at no cost, which is about what I would expect.  Since individual needs will vary - what rate constitutes a rain day, a snow day, a hot day, a cold day, etc. - I can see where this might be onerous.  That said, the Australian Board of Meterology seems to be providing this data.  If such a query could be constructed that could extract this data, it is beyond my capabilities.   I would gladly work with anyone who might be able to do the proper programming that would incorporate various user specific filters - assumes Access using csv data.

Does your company collect rain day data - actual days lost to rain on its various projects?  If they do, it will be useful when trying to validate the models.  Whatever we construct may not have any predictive value if it cannot be properly validated.

Rafael Davila
User offline. Last seen 4 hours 33 min ago. Offline
Joined: 1 Mar 2004
Posts: 5229

Dennis,

Definitively I find it onerous for any owner to require each construction contractor to do this tedious research by themselves. Washing their hands and throwing all the burden on the contractors makes it impossible to make any claim that under other rules would not be onerous, rules our parents used decades ago.

Too much theory but disconnected from the real world, so disconnected that I fear most of the practice that is based on averages is flawed. Using average number of rain days per month will overwhelmingly yield average years and will not take into account the un-average. But the un-average happen every four years and must be considered.

If you end up modeling only average monthly conditions the calculations will miss when near critical activities become critical because of days lost to rain. This is the reason why Monte Carlo is used, because if we only use averages the statistic will be wrong.

I believe there is a possibility 90% of suggested procedures are flawed by missing this point, an error of 50 years ago, if history repeats again it would be a double shame if an error using Monte Carlo models that were developed to prevent it.

The following illustrates the data I am looking for, I need it just for the 12 months of the year for the last 30 years, not much 360 tables. Missing intermediate values between Days >= 0.10 and 0.50 so it falls short of my needs. Will consider looking at the daily reports, 360 days x 30 years, just 10,950 reports for every town a contractor does business, a piece of cake. Well every other contractor bidding on the same job should be doing it.

 photo PRData_zps4002a77a.png

RE: It might be useful to compare our individual results. I am using the hourly precipitation data from NOAA for San Juan International Airport for the date range of 01Jan67 to 10Aug12 (~45 years). If there is an Excel maven out there that could automate the totaling of number of days per month (see below - cut and paste between programs does not always work well). In this case, there are 11 days for Jan67, 6 days for Feb67, 4 days for Mar67, and 4 days for Apr67.....This has to be done for each month of each of the 45 years. This becomes tedious. If anyone has suggestions to simplify the process, I would appreciate it.

Maybe using database software all the data can be imported to a single database file and a single query can do it. I find it difficult to understand why the files are not available in a consolidated CSV format for the public to download all or part of it. In a tabular format easy to consolidate and query. We have hundreds of universities and research centers, not to mention hundreds of thousands of contractors that might also need the data, maybe we are looking in the wrong way, or maybe someone is profiting by selling the data, data that shall be of easy access and free for the general public. Because the NOAA reference is in almost all Federal and State Government contracts it is supposed to be a piece of cake, we must be doing something wrong on how we are pulling down the data.

Best Regards

Rafael

Dennis Hanks
User offline. Last seen 8 years 1 week ago. Offline
Joined: 17 Apr 2007
Posts: 310

Rafael/Stephen:

My suggestion would be to use all available data and then look for trends, as opposed to imposing arbitrary trends.  My analysis of the Lake Charles, LA data showed no discernable trend (I will go back just to be sure).  The data seemed to be random enough (within seasonal limits) to assume 'normalcy' - not that I am a statistician, so you can take that statement with a grain of salt.

It might be useful to compare our individual results.  I am using the hourly precipitation data from NOAA for San Juan International Airport for the date range of 01Jan67 to 10Aug12 (~45 years).  If there is an Excel maven out there that could automate the totaling of number of days per month (see below - cut and paste between programs does not always work well).  In this case, there are 11 days for Jan67,  6 days for Feb67, 4 days for Mar67, and 4 days for Apr67.....This has to be done for each month of each of the 45 years.  This becomes tedious.  If anyone has suggestions to simplify the process, I would appreciate it.

 

 

1967504250
1307
418
711
981
1617
1813
2114
2213
2318
2423
2524
2630
2293
213
911
1634
2447
2728
28126
3146
316
616
714
1570
485
112
1124
1413
2513

 

Rafael Davila
User offline. Last seen 4 hours 33 min ago. Offline
Joined: 1 Mar 2004
Posts: 5229

Dennis,

I edited a prior posting and included the following for which I would like to hear your comments.

  • As you can see I am questioning myself about the threshold limit to define what will predict days lost to rain as well as the use of average monthly days lost to rain instead of using a distribution. Using only averages can led us to the flaw of averages as extreme years will be ruled out in our Monte Carlo model if we limit the model to the use of averages.

Stephen,

I agree with you and maybe because linear extrapolation can yield unrealistic results I would assign more weight to the last ten years. Perhaps a weight of one to the first 20 years and a weight of 2 to the last ten, making it as if 40 years.

I prefer clear rules for contract terms and leaving the use of probabilistic models to the party who takes the most risk, your suggestion makes much sense, if there is a trend it will take it somehow into consideration if not nothing will be lost. 

Regards,
Rafael

Rafael Davila
User offline. Last seen 4 hours 33 min ago. Offline
Joined: 1 Mar 2004
Posts: 5229

Dennis,

Absolutely no issue at all with the use of San Juan, you make the call.

http://www.srh.noaa.gov/sju/?n=climo_san_juan

The average yearly rainfall for San Juan is 70.78 in versus 83.9 for Dorado, 15 miles away. For Ponce it is
62.28, 35 miles away from San Juan. For Lajas it is a mere 47.98 in, about 15 miles from Ponce. Our mountains can make a difference in a short distance.

It is not just about average precipitation but also about the number of rain days per month above certain threshold. I was looking for Dorado because the threshold used to predict days lost to rain if used wrong can end up predicting you will barely be able to work in Dorado while for experience I know it is not that much.

Sometimes it rains torrentially for 1/2 hour and suddenly you get bright sun, if the soil is granular we can continue working if clay 15 minutes of rain might be enough to send us home. We do not run out of our jobs at the first sign of rain, now we use our iPhone and can see the radar. Our Monte Carlo models do not have access to the radar but we got to estimate impact somehow.

If you decide to go with San Juan I will make the switch so communication of our numbers is easier. It might be that for San Juan there are older statistics available. I was looking for a location that I know the number of days with rain above 0.1 in is high, though maybe it is not that different.

Best regards,

Rafael

Stephen Devaux
User offline. Last seen 17 weeks 6 hours ago. Offline
Joined: 23 Mar 2005
Posts: 667

Rafael, one thing to keep in mind is that the ten warmest years on record globally have been since 1997. Since (1) warmer oceans evaporate more and (2) a warmer atmosphere holds more moisture, it is reasonable to assume that rainfall patterns may change.

If I were you, I wouldn't just take the averages for the past 30 years -- I'd also see if there has been a steady upslope since 1997. If there has, I would see where that trend was leading through the duration of your project and adjust accordingly.

Fraternally in project management,

Steve the Bajan 

Dennis Hanks
User offline. Last seen 8 years 1 week ago. Offline
Joined: 17 Apr 2007
Posts: 310

Rafael;

Is there a reason for not using the NOAA data for San Juan International Airport.  Wikipedia says Dorado is only 15 miles away.  Is there some geographical feature that would invalidate the data?  I am currently downloading hourly precipitation for SJIA from  01Jan67 to 01Aug2012 (big file).  Will probably not do anything with it, just was curious.

I hope this forum can develop a methodology for forecasting weather delay.  Whatever we come up with will only be a rough approximation, but it will be better than what we currently have IMO.

http://gis.ncdc.noaa.gov/map/viewer/#app=clim&cfg=cdo&theme=precip&layer...

Rafael Davila
User offline. Last seen 4 hours 33 min ago. Offline
Joined: 1 Mar 2004
Posts: 5229

I agree with Mike and would like to add.

  • Don't forget it also rains during non work hours during regular work days.
  • At times when it rain at night does not matter, at times rain at night matters for a couple of days even if it does not rains again.
  • If you work two 8 hours shifts the probabilities of being impacted by rain are greater than if you work a single shift.
  • If it rains 3 consecutive days the soil get saturated and might have a greater impact than if it rains on alternate days.
  • We have several weather patterns; rain due to wind blowing inland during the day, from sea toward the mountains and rain due to low pressure systems some coming from the East during June and rain systems due to cold fronts from the west during December. These systems can get our soils saturated and will have a greater impact than rain on alternate days.

It does not matter if the parties share or not the risk of weather patter under clear rules, rain will still happen and will have an impact. It is very complicated but shall not stop us from looking for better methods to estimate impact.

As you can see I am questioning myself about the threshold limit to define what will predict days lost to rain as well as the use of average monthly days lost to rain instead of using a distribution. Using only averages can led us to the flaw of averages as extreme years will be ruled out in our Monte Carlo model if we limit the model to the use of averages.

Mike Testro
User offline. Last seen 4 weeks 2 days ago. Offline
Joined: 14 Dec 2005
Posts: 4418

Hi Rafael

Don't forget it rains at weekends.

Best regards

Mike Testro

Rafael Davila
User offline. Last seen 4 hours 33 min ago. Offline
Joined: 1 Mar 2004
Posts: 5229

With the purpose to start exploring the procedure until I find what data is necessary I created the following fictitious data. I learned that there is a possibility that 30 years is not much for the distribution statistics. I believe it might be that rain is not so random as to make it a normal distribution. It is possible some months we experience dry weather and others a rain pattern and that the occurrence of in between is not so common, only the observations will tell us. Sometime we have "el niño", the boy, and sometime we have "la niña" the girl. Perhaps these shall also be considered into the equation.

The most arbitrary value is on what amount of rain will represent a good predictor of days lost to rain. My fear is that it can be manipulated for the benefit of either side.

 photo raindistribution_zps885176f1.png