scoring

Levels 1, 2 and 3 on the UFE and why you should care

Sorry guys – this was supposed to be published last week but there seemed to be a nasty UFE-hating bug that got in the way. Without further delay, here you go:

As mentioned the other day, scoring works differently and the pass/fail determination is different on the UFE. I said that you need to demonstrate both depth and breadth in order to pass the UFE. I also mentioned that secondary indicators are worth zero, except in certain rare circumstances. Today we explain.

The UFE is marked using the Level 1-3 system. Where your response is marked for three different things.

Level 1 – Your overall score

Like SOA, Level 1 takes your overall score and compares it to other scores (an average of some kind). If you fall below the cutoff you will not pass and if you are above the cutoff that’s great but you aren’t out of the woods yet. Your response will be examined at Level 2.

Your goal, therefore, is to score your overall UFE score above the cutoff to pass this.

Most people (something like 98% last time it was disclosed) fail at Level 1. Keep this in mind when you are worrying unnecessarily about Level 2 and 3.

 

Level 2 – Demonstrate Depth

This level applies to PMR and Assurance ONLY.

This level requires that you demonstrate enough depth on enough PMR and Assurance indicators. What does depth mean? The number of Competent (C-level) responses you give on primary indicators. Therefore if you score only RC on all PMR indicators you will not pass at Level 2. Even if you score all Cs in Assurance. Nervous now? Well don’t be, look up at the bold text right above 🙂

How many Cs do you need to achieve in order to demonstrate depth? It’s not disclosed but rumours have it that it’s two or three. This may sound like a lot to some but again, read the bold text above!

Alright, so you’re having a good day, you’ve survived the Level 1 cutoff and you’ve got enough Cs in PMR and Assurance. Can you walk away with that coveted UFE Success letter? No, you’ve got one last hoop to jump through.

 

Level 3 – Demonstrate Breadth

This level applies to GSRM, Finance, MDM and Taxation ONLY.

You must score sufficiently on all four of the above indicators in order to be successful at Level 3. What does sufficiently mean? It means you have to score RC enough times on each of the competency areas above.

Again, we don’t know how many RCs that means but I suspect it’s one each. Once again, if you’re starting to get nervous, I’ll remind you that something like 98% of people fail (when disclosed) at Level 1.

 

Level 3 Faint Hope Clause

Finally, we come to the secret behind secondary indicators. Under one very rare circumstance, you can pass the UFE because of a secondary indicator where you would have failed otherwise.

If you fail on Level 3 above, and in one of the primary indicators (for example Tax) you did not score any RCs BUT you managed to score Competent (C) on a Tax secondary indicator you will pass.

I’ll remind you that secondary indicators are otherwise worthless. I don’t recommend ever wasting your time on them but as a good UFE Blogger it’s my responsibility to inform you about this.

 

And that’s how scoring on the UFE works.

The Key Takeaways:

  • Score well enough overall (PQs included here)
  • Get enough Cs in both Assurance and PM
  • Get enough RCs in Tax, MDM, Finance and GSRM
  • Don’t bother with Secondary indicators

Understanding the School of Accountancy SOA Exam Basics

Today, for those in Ontario.

The School of Accountancy (SOA) final exam is different from the UFE. It’s also similar in some ways, but it’s different in ways that you must understand to be most successful.

Here is an introduction to some of the big differences:

SOA is (more) about breadth, UFE is (more) about depth.

  • The SOA exam does not have the same three levels of marking that the UFE does. This has important implications for scoring such as the fact that you don’t need to hit every indicator to pass the exam.
  • The SOA has favoured breadth over depth in the past. This means that it’s important to hit everything at least on the surface. Skipping or messing up an indicator can really harm your score which is often caused by not managing your time properly. I had this experience during SOA when I messed up an important indicator in SOA2 (the practice exam) which landed me in the low deciles.

SOA is scored differently than the UFE

  • The UFE has primary and secondary indicators, the SOA has Diagnostic and Summative indicators. This is not just a difference in name.
  • Diagnostic indicators are not worth any marks and are there to help you understand what you need to improve on in your case.
  • Summative indicators get their value (mark) from the summary of the diagnostic indicators and this is what is worth points.

How SOA scoring works

There are five scores you can receive on an SOA exam per indicator, same as UFE, but their value is different from the UFE.

SOA UFE
NA (Not Addressed) 0% 0%
NC (Nominal Competence) 25% 0%
RC (Reaching Competence) 50% 50%
C (Competent) 100% 100%
HC (Highly Competent) 100% 100%

From the above table you can see that SOA gives you a little crutch in the NC score on each indicator.

What does NA, or RC or C mean?

The requirement for each will vary by case and this is part of the process of learning case writing. But in brief,

NA = You missed the indicator, didn’t know to write about it or didn’t allocate the time to do so.
NC = You saw the indicator but your analysis was superficial or you didn’t use many case facts.
RC = You provided some depth in your analysis and used case facts.
C = You provided a strong analysis which was correct and used case facts to support it.
HC = You provided a very strong analysis which was correct and used case facts very well to support it.

You can see here something that you’ll be hearing a lot about: using case facts. In SOA and UFE you cannot get away with just spitting out technical (usually worth zero), you’ll have to integrate the facts from each individual case into your analysis.

SOA Scoring Example

SOA Scoring Example

Using the example above, you can see each component that make up a score for a single competency indicator.

Each simulation will have multiple indicators, and you’ll be writing multiple simulations, everything is added up and you’ll receive a pass or fail (the passing mark is not disclosed publicly).

 

What else are you curious about with SOA? Leave a comment and we’ll tackle things here as we go!

Secondary Indicators on the UFE

One of the things people are searching around on this site a lot for are secondary indicators of competence. I was actually a bit puzzled as to why so many people are looking for info on secondary indicators but then I saw “ufe are you supposed to talk about secondary indicators” as one of the searches and it clicked.

Since many if you want to know, here’s some info on secondary indicators:

What are Secondary Indicators?

The UFE Report states that secondary indicators are “… indicators of competence [which] answer the question: “What other issues could a CA raise?” Although such issues are valid, it is not essential for a competent CA to address them.”

In general, you should only be responding to primary indicators and responding to secondary indicators is not an advised strategy. The reason being is that secondary indicators are not worth any points on the UFE except on very borderline responses in very specific circumstances. Secondary indicators were more frequently used in the past in order to evaluate ranking but are used much less now with ranking evaluated within the primary indicators.

Secondary indicators are scored as NA, NC or C vs. primary indicators which also have RC and HC. This means that a secondary indicator is worth nothing unless you obtain a C on it, and even then, as discussed above, only in specific circumstances. Therefore, I think it’s safe to say that you should avoid tackling secondary indicators!

Here is a breakdown of how secondary indicators were tested on the past four UFEs.

2008 UFE: 1 Tax, 1 GSRM
2009 UFE: 1 GSRM, 1 Finance, 1 Tax
2010 UFE: 1 Tax, 1 Finance, 1 GSRM
2011 UFE: 1 MDM, 1 Finance, 1 Tax

So it seems like three is the standard recently.

How do you tell the difference between primary and secondary? In most cases primary indicators are specifically asked for by the client or your boss while secondary are mentioned in passing. Don’t ignore a specific request by a client or your boss because you don’t deem it as important, secondary indicators won’t show up as specific requests but as incidental requests “if you have time…”

Now get back to those primary indicators!

UFE Pass Rates

Something a lot of candidates are always searching for is the UFE Pass Rates. The UFE Pass rate used to be published in the annual UFE report but this practice was stopped in 2009 and the pass rates have not been released since.

Here is what the pass rates were like when they were known:

  • 2003: 65.5% (1,707 Successful out of 2,607 writers)
  • 2004: 74.5% (1,908 Successful out of 2,561 writers)
  • 2005: 74.0% (1,862 Successful out of 2,516 writers)
  • 2006: 79.3% (L1: 81.5%, L2: ~96.3%, L3: 97.8%) (2,136 Successful out of 2,694 writers)
  • 2007: 74.6% (L1: 71.6%, L2: ~97.1%, L3: 98.2%)* (2,701 Successful out of 3,117 writers)
  • 2008: 71.7% (Individual Level results don’t appear published) (2,640 Successful out of 3,682 writers)
  • 2009: 3,127 Successful
  • 2010: 3,046 Successful
  • 2011: 2,857 Successful

* There is clearly an error here but this is as it was published. I assume L1 should have been 74.6%

Note: I obtained the pass rates from the UFE Reports but the number of successful and total writers I gathered from numerous sites on the web, not all of them official. I had to calculate a few of these numbers based on the pass rate as well, so chances are it’s not 100% accurate but should be close. If anyone has a reliable source for any number up there that is wrong please let me know!

As you can see, the pass rate used to be in the 74% – 80% level so overall the odds are in your favour by a decent margin!

The importance of Levels 1, 2 and 3

We discussed previously how UFE scoring works and referred to the different levels of passing before. It’s important to understand each level in depth so here’s the depth.

The UFE is marked based on a decision model with 3 different levels you must pass in order to pass the exam.

Level 1

The 2011 UFE Report defines Level 1 as the following on page 2:

The response must be sufficient, i.e., the candidate must demonstrate competence on the primary indicators (Level 1). In assessing sufficiency, the board considered the number of times that a candidate achieved “Competent” and/or “Reaching Competence” across all primary indicators (both specific competencies and pervasive qualities).

This is the total score level. If you did not pass at level 1 this means that your total exam score across all competencies was below the cutoff. Since only RC, C and HC give you a score, it also means you did not have enough RC, C or HC scores. Although the CICA no longer releases how many candidates failed at each level, when they last did the vast majority failed at this level.

If you passed at Level 1, that’s great, but you’re not out of the woods yet. You must also pass Level 2 next. This is different from Ontario’s SOA which is based on a total score only.

Level 2

The 2011 UFE Report defines Level 2 as the following on page 2:

The response must demonstrate depth in the areas of Performance Measurement and Reporting and Assurance (Level 2). In assessing depth the board considered the number of times that a candidate achieved “Competent” in each of the Assurance and Performance Measurement and Reporting primary indicators.

This levels tests your ability to show sufficient depth in PMR and Assurance which are given the two highest amounts of weight on the exam. Notice that to pass this level you need a sufficient amount of competent (C) level scores across these two indicators. How many C marks do you need? Unfortunately that is not disclosed. In the 2011 UFE, seven Assurance primary indicators were tested and seven PMR indicators were tested. I’ve heard rumours that it’s as few as two and as many as four or five. If I were betting I would say three but it’s anybody’s guess – What rumours have you heard?

So, if at this point, your overall score is sufficient (passed Level 1) and you’ve attained enough C in PMR and Assurance (passed Level 2). There is still one more hoop to jump through in order to pass the UFE.

Level 3

The 2011 UFE Report defines Level 3 as the following on page 2:

The response must demonstrate breadth across all areas of the Map (Level 3). In assessing breadth the board considered the number of times that a candidate achieved “Reaching Competence” across primary indicators in each of the specific competency areas, except for Assurance and Performance Measurement and Reporting. If a candidate failed to demonstrate breadth on the basis of the primary indicators, the board considered the information provided by the secondary indicators for the deficient competency area.

This level looks at the competencies besides PMR and Assurance. You must score a sufficient number of RC or higher marks in each of GSRM, Taxation, MDM and Finance. In the 2011 UFE, each of these competencies was tested as follows:

  • GSRM: 2 indicators
  • Finance: 3 indicators
  • Taxation: 3 indicators
  • MDM: 3 indicators

Again, the actual amount of RCs you need in each is not disclosed but my gut feeling is that it was probably one in each case. What do you think? In any case, this level ensures that candidates can’t skip any particular competency. So if you are terrible in tax, you’ll still need to demonstrate an RC level somewhere in tax.

As I’ve mentioned before, there is a last-hope sort of mechanism built into Level 3 which allows you to pass if you did not score sufficient RCs in a competency on the primary indicators but were able to show competence (C or higher) in a secondary indicator of the same competency. For example, if you did not score sufficient RCs in Taxation primary indicators, but scored C or higher in a secondary indicators, the board may pass you based on that.

Simple, eh?

What about the Pervasive Quality (PQ) Indicators?

PQ indicators add to your score at Level 1 and that’s it. You do not need to score a certain amount of RC or C in the PQ indicators in order to pass but they help to elevate your level 1 score.

Any questions?

How does scoring on the UFE work?

One of the differences between SOA and the UFE I’ve mentioned is that the scoring is different in a number of ways. If you’re from Ontario here are some of the changes you can expect, if you’re not then consider this of general interest.

What’s the same?

  • You’ll still use Not Addressed (NA), Nominal Competence (NC), Reaching Competence (RC), Competent (C) and Highly Competent (HC) to score your simulations.
  • Scores of RC are still worth the same amount of 3 point each.
  • Scores of C and HC are worth 6 points each. Therefore there is no advantage for the common candidate to strive for better than C. I believe this only becomes a factor if you’re going for a medal so unless you’re pretty certain your medalist caliber this is not something I’d be striving for.

 

For Ontario Students: What’s different from SOA?

  • Scores of NC are no longer worth anything and receive a score of zero which is the same as NA scores. This removes the crutch that SOA provides of giving you points just for identifying an issue.
  • As mentioned previously, there are no more diagnostic indicators and they are simply rolled up into a single primary or secondary indicator.

 

What are Primary Indicators?

Straight out of the CICA UFE Reports, primary indicators are defined as:

Primary indicators of competence answer the question: “What would a competent CA do in these circumstances?” If the issues identified in primary indicators are not adequately addressed, the CA could, in real life, be placed in professional jeopardy or could place the client in jeopardy.”

In other words, these are topics that you must identify and discuss in order to score adequately. These are equivalent to summative indicators for SOA.

Again, out of the CICA UFE Reports, secondary indicators are defined as:

Secondary indicators of competence answer the question: “What other issues could a CA raise?” Although such issues are valid, it is not essential for a competent CA to address them.

In other words, these indicators are not worth any points except in one last chance type of situation. Secondary indicators are only reviewed if you do not pass sufficiently at Level 3. As we’ll discuss in tomorrows post, not passing at Level 3 means that you did not score sufficient RCs in Tax, Management Decision Making (MDM) or Finance competencies. On that review, if you were able to score a C in a secondary indicator for which you did not have enough RCs in primary indicators you can pass. For example, if you did not score sufficient RCs in Tax primary indicators but were able to score C in a secondary indicator which balances it out you could still pass. As you can see, this is a pretty long shot so you are better off not spending too much time on secondary indicators.

That’s the basics of marking. We’ll dive in to how to decide what ranking to give, how passing the exam works and more in future posts.

Pin It on Pinterest