There’s been some people searching around this blog for tips on how to mark a case and I’ve been asked to discuss marking a little. With less than two weeks until the UFE you should start, at this point, trying to mark simulations pretty realistically. Not too easy and not too hard. Usually candidates tend to mark too hard to be on the safe side but there’s of course always the danger that we’re being too easy. As I’ve mentioned before, there’s no definitive information from the CICA of exactly what consititutes an RC, C or a HC but they give you some pretty good hints. Today we’ll go through marking a real candidates response here using the the UFE Report. As you may already know, there are marking guides out there that make this a little easier but I think it’s a good idea to do it the old fashioned way once in a while. I do believe it helps a little more with understanding what’s required.
The case we’ll look at is 2009, Paper II, Simulation I – Memorial City School Board. I’m including the valid parts of an actual real response. Please download the accompanying two files and follow along there to see how this specific case was marked.
Word Document Download: 2009-P2-S1 + Excel Quant Download: 2009-P2-S1
Here is how I would mark this simulation.
Primary Indicator 1 (Finance): The candidate compares the PPP option with the alternative of the CSB constructing and operating the school itself.
From the solution, you can see that this indicator has both a quantitative and a qualitative element.
You had to compare Option 1 : The school board builds the school and had to compare that to Option 2: PPP with Fox.
From the solution you can see that there are 6 items in the calculation of Option 1 and 5 items in the calculation of Option 2. It’s critical to read the Board of Evaluators comments which present good information about how this was marked. From the comment right below the calculation you can gather pretty strong evidence of a few things they were looking for.
- A Net Present Value calculation was neccessary
- Large items mattered, particularily rental revenue
You can therefore assume that in order to get to a C you had to
- compare both options,
- include the large items
- include rental revenue
- do a Net Present Value calculation
At the end of the Primary Indicator you can see that in a Competent response, “The candidate prepares a reasonable quantitative assessment of the options” so the only thing we need to still determine is what constitutes reasonable in this quant. We must use some judgment and considering this case, I would say that reasonable means the 3 clearly large items from each Option should be compared including the rental revenue.
What did our example candidate achieve?
- Our candidate included three items from the PPP agreement and five items from the CSB options. They attempted a NPV but sometimes the numbers were off. They got all the large number items including the rental revenue. The quant was well documented what they were trying to achieve and what they were including. Based on this it appears that they knew what they were doing and this is a reasonable quantitative analysis. The numbers have to be reasonable, they need not be correct, except often with tax you need correct numbers.
Next, there is a qualitative element to this indicator which is a discussion of the three options. The solution has the following three things discussed:
- Board of Trustees should consider it may be exposed to cost over runs if it chooses to build the school
- Funds would have to be disbursed up front if they choose to build the school vs. not paid up front with Fox
- CSB would be exposed to risk of inflation and would have to assume the extra cost over the years
- Even a small difference in the assumptions could yield significant differences in the results, assumptions should be confirmed
- Advantage of going with Fox – Rental revenue may be higher than if CSB ran the school
- Reliability of the PPP numbers vs. the reliability of the CSB option numbers
The board mentioned that candidates were rather weak on the qualitative aspect of this indicator but still 59.1% of candidates achieved competent. This leads me to judge that you probably only needed to discuss two of the items above in order to reach competent.
What did our example candidate achieve?
- This candidate discussed item 1, discussed item 2 and mentioned something not on the above list, the responsibility of Fox for collections which is outside of the core competency of CSB. This last point I thought was good even though it’s not on the solution so I considered it valid. Always use your judgment. Finally there was a strong overall conclusion of which choice to go with and why. This is a competent response based on my criteria.
Overall Indicator Score: Competent – The candidate did a reasonable quant and discussed, using case facts, at least two options.
For RC, the candidate had to attempt (which I read as include at least two valid items in each option) a quant AND discuss at least two qualitative items. OR, the candidate had to prepare a reasonable quant (same criteria as above) but without a qualitative analysis.
Primary Indicator #2 (GSRM): The candidate discusses how some of the risks related to the PPP can be managed.
The solution mentions the follows risks as things that could be discussed:
- Agreement – Terms and condition,
- Agreement – dispute resolution,
- Agreement – Discussing Fox the company.
- Building – Design,
- Building – Construction,
- Building – Value
- Use by Fox in off hours
- Quality of Services – Maintenance
- Quality of Services – Food and beverages
So as you can see, there was a wide range of issues and for RC the board expected the candidate to discuss some risks while for C the board expected candidates to discuss some risks and ways that they could be managed.
So what we must decide is how many does some constitute. When in doubt, I always assumed it was half of the ones mentioned in the solution. So out of nine issues here, I would say that four needed to be discussed to constitute some. In reality, it might have been three or it might have been five but we can’t know and half is always my benchmark for a C. For a C, all four needed to have mitigation/ways to manage these risks.
How did our example candidate fare?
- This candidate discussed item 7 and provided a suggested mitigation (outline restrictions)
- The candidate discussed item 8 and 9, the quality of food/service but did not provide a good means to mitigate this
- The candidate discussed allowing Cola in the schools and suggested they write into the terms restrictions
- The candidate discussed item 3 and suggested that CSB have a clear plan in place to deal with the possibility
- The candidate discussed item 5 and suggested that materials allowed in construction be specified
And so forth, a few other items were mentioned but this candidate had discussed, using case facts, more than four issues and provided means by which they could be mitigated or managed. This was a competent response.
Overall Indicator Score: Competent – The candidate discussed at least four issues and provided a means to manage them.
Primary Indicator #3 (Assurance): The candidate explains how the CSB will ensure Fox delivers on all the promises it has made with regard to the PPP.
This indicator is a procedures indicator and the solution discusses the following items:
- Rental revenue
- Volume discount on food contracts with QNI
- Vending machine contract with Right Cola
- Repairs and maintenance
- Minimum government standards
- Fox Property Management Corp.
For an RC the candidate had to identify some of the procedures the CSB could perform to ensure obligations were met. For a C the candidate had to discuss some of the procedures.
So it would appear that the same amount had to be either id’d or discussed. As before, I always like to stick to half so in this case, three procedures had to be id’d for an RC and they had to be at a discuss level for a C. What does discuss mean? If you read the Board’s comments, they expected candidates to identify the key promises made and then provide procedures that would provide assurance over these areas. So the standard for this appeared to be the validity of the procedure to the case. When marking these procedures, you would have to use your judgment to decide whether it is a more general procedure (not using specific case facts) or whether it is a good procedure (something that could be done, is valid and uses case facts)
How did our example candidate fare?
- They discussed rental revenue, which is item 1 above, and suggested a log of room use should be kept. The candidate went a little further with this one than they needed but reviewing a log book is a valid procedure. I also recommend that procedures always be stated in terms of the risk they are covering off. This gets ‘discuss’ level.
- The candidate did discuss the price/quality of the food (covered under item 5 above) and suggested that surprise inspections be done and samples be sent to labs. I kept this at ID level only.
- The candidate discussed repair and maintenance (item 5 above) and did suggest keeping a log. As the board mentioned, this was okay but would have been better to discuss more specific items. I kept this at ID level only
- The candidate began a discussion about vending machines but ran out of time. However, the one sentence was good enough for ID level.
Overall Indicator Score: Reaching Competent – The candidate discussed one procedure and Id’d 2-3 more.
Primary Indicator #4 (PQ): The candidate discusses other options or issues that should be considered before a final decision can be made
This was a tough PQ and an RC required that the candidate ID one more option or issues that should be considered before a final decision. A C required that a candidate discusses another option.
The achieve ID the candidate had to at least consider that more options should be considered but failed to recommend anything to CSB. To discuss, a candidate had to realize that other options should be considered and provide a recommendation.
This candidate actually did perform well on this PQ. In their overall conclusion they recommended that the agreement be reviewed and touched on some discussion of splitting up the responsibilities with the school between CSB and Fox.
Although I was tempted to give out a C for this indicator, I looked at the numbers and only 4.1% of candidates achieved a C so I felt like although this candidate touched on the item, it was probably not a strong enough discussion to obtain a C. Maybe in reality it was but I sided with caution this time.
I hope you found that helpful. Keep in mind that this is a stronger response and most candidates will have some combination of RCs and Cs and an NA or NC on the PQ. The key take away here is how you should go about marking a case using the UFE Report.
Debrief that sucker! Happy Studying!