Qualification achievement rates 2024 to 2025

Hi Sarah

Yes you are right on this. A change in the methodology means that you now have 4 months less for learners to return from a break and them not to be counted as a penalty leaver within the QAR. 

I assume (and we know the folly of assumptions) is that the DfE have looked at the proportion of learners who come back in that window and have decided that this would not make a significant difference to achievement rates.

Thanks

Paul

Hi Paul

Thank you for your response.

The addition of 'continuation of break record' suggests that they can continue to be recorded as a break in learning at that point without being deemed as withdrawn.

The change in the 4 months won't have a huge impact as it is rare we have these kind of timescales on breaks in learning, however it looks to us like the rule has changed around having to withdraw apprentices who are on long term breaks in learning.

Thanks

Sarah

Oh, heck, yes, of course this will have to happen if they're not using R04 any more! Hadn't even occurred to me...

Sarah Hudson you DO NOT have to mark them as withdrawn though, they can stay as a BIL and when they DO come back and achieve they will count positively against you in that year.

Wait, no, hold on...

wait wait wait...

Have we not go this backwards...

We also treat learning aims as withdrawals in the QAR methodology where they have a Completion status of 6 (‘Learner has temporarily withdrawn from the aim due to an agreed break in learning’) and where either: 

• they do not have a corresponding restart record in the same funding year 
• they do not have a corresponding restart record or continuation of break record in the following funding year. 

For example, if you recorded an aim with a planned end date in July 2024 that had a break in learning beginning in April 2023, this will be a withdrawn aim in the 2024 to 2025 funding year if there is no restart record or continuation of break record in either:  

•  the R14 ILR return in 2023 to 2024

•  the R14 ILR return in 2024 to 2025

This is just saying there needs to be a BIL record in the subsequent year's ILR, I don't think it matters if they haven't come back???

Oh heck, feels like a trip to the Customer Service Desk for me...

My reading of this Steve is that the process is as it was but you just have 4 months less to return. If you are a BiL then you have up to the end of the following academic year to return in order to not be counted as a leaver in the QAR. The difference is that you no longer get the first 4 months of the 3rd academic year to return in now that they are no longer using the R04. 

The process was already inequitable in that a learner going on a break in July would have 16 months to return (now 12) whereas an August break had 28 months (now 24). Obviously most breaks are shorter than a year but suspect there will be some small impact on headline achievement rates as a consequence of this. 

I think I will raise a query with customer service desk, although I don't expect a definitive answer. 

I am with Steve, the addition of the continuation of break record suggests as long as they remain recorded on a break they are excluded.

I will raise with customer services and update here on their response.

We can compare answers!!!

I don't think that is any change from the current process. 

If a learner is on a break in learning in 23/24, then that break in learning continues to be submitted in corresponding years until either the learner returns and restarts their learning, or you report the learner as having withdrawn . 

If the learner has not returned by R14 24/25 then this will be counted as a withdrawal in the QAR.

Where a learner is retrospectively changed from a break to a withdrawal in 24/25 but with a 23/24 learning end date, then this learner stops being included in the ILR return. In this instance learners get added into the QAR as withdrawal and labelled as an overdue break in learning and added to hybrid end year based on planned end / actual end +1 year. 

My reading of that is that, aside from the R04 change, this is the same as it was previously but interested to see what the combined response is from the service desk is. 

Ah, but, not everyone *does* roll over their BILs into the following year's data (more fool them, particularly if they're Apps), there's certainly no Rule that says you must and, as far as I know, with most student record systems, it's a pain in the bum to do it...

Let us see what they say though.

So after re reading this many times I now get that the fact that it has the continuation of break record stated actually doesn't make a difference to the 2 year rule, they would be either returned, withdrawn or continued on break anyway in that time, it is just stating the obvious. The only difference as Paul said is the 4 months. I did however already raise the query with DfE and this is what they came back with:

Every time an aim is returned as a break in learning, the 2 year timer resets.
 
So if an aim is returned as a break 3 years in a row, but then not returned at all for 2 years after that, then the aim will become overdue and count as a withdrawal.
 
However, it's also worth noting that although technically an aim can keep getting returned as a break, doing so repeatedly for longer than breaks are expected to last for may trigger an investigation by DfE.

What I don't understand is why this isn't detailed as an actual funding rule, they set no limit however you might be investigated if it goes over the 2 year 'rule' that isn't a rule.

We don't allow breaks to go past the 2 years anyway as the likelihood of a successful return for longer breaks diminishes as time goes on but it is something that we have always questioned as it isn't very clearly written.

You'll be delighted to hear that my response from the Service Desk only talked about AAF and not QAR, due to the usual lack of reading comprehension, so I've sent it back to them to look at it again...

My interpretation of this is I think the same as Steve Hewitt - that provided BILs are rolled over and continue to be reported as a break until they return or withdraw, they won't be treated as an overdue break (and thus withdrawal) under the QAR methodology.

I actually wonder if this has always been the case...and the only change is that this has now been clarified by the addition of the reference to continuation of break records in the guidance? (aside from the changes to not use R04 data of course).

Overdue breaks is the bit of the QAR that I've never quite mastered, my internal calculations are normally spot on, except for where I have expected the odd long-term break to be treated as overdue and counted as a withdrawal under QAR, but it hasn't. As a provider, we always roll ongoing breaks into the current ILR year until they return or withdraw, and if converted to a withdrawal, we would include them in the ILR data for the year when that decision took place, which is the year they then impact the QAR (reporting year becoming the hybrid year end if it's later than the planned or actual end date).

I had assumed the intention of overdue breaks in the methodology was to prevent providers manipulating QAR by just leaving learners on a break instead of withdrawing them, but the AAF would now cover that, as your percentage and duration of breaks would flag you if you tried to do that, but perhaps that is no longer (or was never) the reason for it.

Yes, just got basically the same response back as Sarah Hudson:

An aim marked as a BIL means the aim has an actual end date (the day the learner left to go on a break) and that the aim is closed.

The system therefore awaits a new return to learning aim to be submitted.

Every R14 that the BIL aim is returned, it rests the 2 year timer for when the system expects a return to learning aim to be submitted by.

However, if an aim is repeatedly returned as a planned break, exceeding the length of time planned breaks usually take, this can trigger an investigation.

This is to avoid learners being left as an infinite BIL

Obviously have asked them to define "usually"!

An Update:

As long as an aim on a planned break is continually reported (included in your ILR with a completion status of 6) that aim will not be treated as a withdrawal for QAR purposes.

Meaning the BIL aim will always be excluded. It is up to accountability colleagues to decide if a situation calls for an investigate into instances of misreporting and statistical gaming.

Those reporting continuous breaks accurately will not be adversely affected.

I think that response contradicts what is in the spec, and we think our R10 data shows no change in process for BiL. If they do not return by R14 of the year after they went on a break, then they get treated as a withdrawal in the QAR. The section on breaks in full below.

Overdue planned breaks

We also treat learning aims as withdrawals in the QAR methodology where they have a Completion status of 6 (‘Learner has temporarily withdrawn from the aim due to an agreed break in learning’) and where either: they do not have a corresponding restart record in the same funding year they do not have a corresponding restart record or continuation of break record in the following funding year If this scenario happens, we will set the reporting year to 1 year after the later of either: the expected end year the actual end year For example, if you recorded an aim with a planned end date in July 2024 that had a break in learning beginning in April 2023, this will be a withdrawn aim in the 2024 to 2025 funding year if there is no restart record or continuation of break record in either: the R14 ILR return in 2023 to 2024 the R14 ILR return in 2024 to 2025 When we treat records as withdrawn due to the situation described they become flagged and described as ‘overdue planned breaks’. An ILR record is a restart record if it has a Learning Delivery Funding and Monitoring Type of ‘RES’ with a Learning Delivery Funding and Monitoring Code of 1 (Learning aim restarted). We match restart records to planned break records by matching the UKPRN, Unique Learner Number and the Original learning start date on the restart or continuation of break record where it matches either the: learning start date original learning start date (to allow for situations where the planned break was itself a restart) of the planned break record for either: Programme type and Framework code for apprenticeships or Learning aim reference for other aims

Good morning all (I'm a little late to this party it seems!)...  

Can I confirm my understanding that a learner who went on Mat Leave BIL (eg) in July 2024 - which was included timely in R12 23/24 now MUST be back on programme before next Wednesday to ensure she is not counted as a Withdrawal?

Where as previously we could have given her until November?

Hi Carrie

Yes, I think that is correct. I'm assuming if you had her in R12 that she would also been in R14 as a break.as well. 

Wow!  So nice of them to land a change on us at the very last second...

The issue I have now is trying to ensure that everyone understands that R14 means "by 31st July"...  that the rest of the 'window' is my tidying and checking time!

I'm not sure if that's what it means, it does state in the above that it'll be treated 'if there is no restart record or continuation of break record in either: the R14 ILR return in 2023 to 2024 the R14 ILR return in 2024 to 2025' so if a learner on a break in learning in July 24, comes back August 25. As long as at R12 and R14 24/25 has a break in learning record, R01 25/26 can have a restart record and it still be counted.

The trick will be making sure that your LMS is including these records. Admiditedly until the data gets published I feel these specs can always be a bit tricky as keeping up with all the rules is just so hard.

Hi Sam

I don't think that is right. The information in the QAR specification is pretty clear on this. In the example of learner going on a break in July 24 they have until July 25 to return. A restart in August 25 would not meet the criteria below.  

Overdue planned breaks

We also treat learning aims as withdrawals in the QAR methodology where they have a Completion status of 6 (‘Learner has temporarily withdrawn from the aim due to an agreed break in learning’) and where either: 

• they do not have a corresponding restart record in the same funding year 
• they do not have a corresponding restart record or continuation of break record in the following funding year. 

Paul, there are now infinite BILs *as long as you keep returning the BIL record* as shown in the responses I got from DfE above. I'm not sure why you're struggling with it? Even the bit you post says "continuation of break record" pretty clearly. Is it because you're not seeing it in your data (and how much can we trust the R10 data to be accurate?)?

I agree with you Steve.

It won't have a huge effect on the QAR, and the BIL kept in check by the AAF. 

Most learners come back from a BIL within a year, however there are some where that is not the case. In the past we've had to make them leavers, however that will no longer be the case.

Honestly, I don't know which way to go with this one, It reads as Steve is suggesting, that providing you return the BIL record, that it will never be overdue, and never be treated as a leaver, that the AAF will be the backstop to ensure we don't keep leavers on BILs to inflate the QAR. 

So why refer to "overdue planned breaks" if the don't exist, surely it should say "We treat learning aims as withdrawals in the QAR methodology where they have a Completion status of 3, if they have a Completion status of 6 we will ignore them". Why all the confusion on which year they went on the break, when they were originally due to finish...

Additionally, why include the field "overdue_planned_break" in the QAR data if the only possible outcome is FALSE. 

This definitely looks like something that needs an official clarification. 

Hi Steve

It is entirely possible that I am not getting this, and appreciate the response you have shared from DfE above, but will explain my thinking below. 

I keep going back to the QAR spec which is the published guidance and therefore should be the source of the truth. In the overdue planned breaks section the first logic is clear to me. If a break in learning does not return in that ILR year or the subsequent year, then they will be flagged as an overdue break in learning and therefore counted as a withdrawal.

My reading of the "continuation of break record" is that this explains the process by which the continued submission of the BiL means that the learner is not counted as an overdue break through year 1 and year 2, but that if they have not returned by the end of year 2 then the earlier logic is then applied.

Again, I appreciate the DfE info that you have shared but that is not what the guidance says and yes, we are seeing some of these in our data.

Infinite BiLs (sounds like one of those recurring MIS staffers recurring bad dreams) would be a significant change in approach which is not communicated in the changes part of the spec but again note your point above on approach through the accountability process. We know where BiLs as QAR exclusions have taken us before so this would seem like a departure for the DfE.

It is a slightly different point, but Sam Bern in the other QAR thread has fed back that late notified breaks to withdrawals with end dates in the previous year have not been acknowledged as an error so possible an issue with BiLs in the data extracts. 

I suppose that leaves two points; can someone from the DfE pick up this conversation and bring some clarity to our discussion – I will also take this to them – and can the QAR people talk to the sector about such changes. I did raise this again at the latest tech user group that we need more conversation and openness on what is the most important metric in the sector. 

In the QAR specification they use this example. Learner goes on a break in learning 23/24. 'Overdue planned break' will be set to true if there is no withdrawn aim in 24/25 or there is no record of their BIL in 24/25. If there is a Break in learning aim returned in 24/25 then 'overdue planned break' is false as it's being returned and the break continues.

I think an issue we're having is this new simplified QAR breakdown which actually makes it slightly harder to anticipate how the data reporting will handle this. The reality is this calculation isn't simple, it's really complicated that uses five years worth of data with different rules about how each aim is treated and counted.

Example below

For example, if you recorded an aim with a planned end date in July 2024 that had a break in learning beginning in April 2023, this will be a withdrawn aim in the 2024 to 2025 funding year if there is no restart record or continuation of break record in either:  

•  the R14 ILR return in 2023 to 2024

•  the R14 ILR return in 2024 to 2025  

When we treat records as withdrawn due to the situation described they become flagged and described as ‘overdue planned breaks’.