Teething issues are to be expected for an exam announced with such short notice but the simulation of the National Law Aptitude Test (NLAT) admissions test to NLSIU Bangalore today was dogged by myriad issues.

As first reported on our live blog earlier today, the simulation of the NLAT was off to an inauspicious start with severe delays and issues to notifying candidates of which actual time slot out of three they would be appearing in.

Apart from such problems that should have been entirely avoidable, there were also the potentially catastrophic, as we have received reports of many candidates trying to (apparently successfully) skirt the proctoring system, as we had predicted they would in our deep-dive of cheating on NLAT yesterday.

And as the Supreme Court has declined to stay the exam (while also not allowing NLS to begin admissions or to release the merit list before the next hearing on 16 September, which will screw with schedules), what happened during the simulation mock could be a more serious problem.

We have reached out to NLSIU for comment on both of these issues and will update the story if we hear anything.

Update 22:30: We have received a statement from NLS about the technical issues experienced by candidates today:

The simulation test was not a mandatory test. All students were given a chance to attend at least one simulation test. In the case of some students where they received late emails about their simulation test, they were given another chance to take the simulation test at 8 pm on Friday.

The helpline was active throughout the day on Friday. However, to assure students who may have concerns about accessing the helpline, staff of more than 115 will be available to man the helplines for technical assistance on Saturday when the NLAT 2020 will be conducted.

With regard to facial recognition issues, candidates have to comply with the instructions given to them in the candidate guide regarding the format of the ID card to be produced during the test.

Same questions, same answers

The simulation mock exam, which included proctoring, took most candidates mere minutes to complete, several candidates told us.

Just like the previous mock exam (without proctoring), it repeated the same 10 questions and the same reading comprehension passage as in the sample paper announced on the exam’s Day Zero (3 September), which most candidates would have looked at or completed at least twice now.

While the FAQ had promised a “Simulation Test of the exact NLAT examination process”, it seems like a missed opportunity at assuring candidates by not having them be able to attempt an entire 40-question, 45 minute proctored exam paper.

Proctoring schmoctoring?

Be that as it may, this simulation may mostly have been intended to be a dry run of the proctoring system.

If so, reviews of the proctoring have not been great either.

DISCLAIMER: None of the below accounts are endorsements of cheating or describe methods that will necessarily work in the final exam, which may have more stringent security measures, in addition to potentially having a post mortem process where successful candidates’ camera footage could be examined more closely. In short, even if the system is not perfect, it is still very risky to cheat and if you’re unethical enough to try cheating in the actual final exam, no one but you bears responsibility for your failure.

Update 14:57: A new section has been added to the FAQs that confirms: “Further investigation post the examination will be carried out with help of audit logs generated on the system, and NLSIU may disqualify the candidate upon such examination”.

“The whole proctoring system was a joke,” said one candidate. They said that despite bad posture, holding up their phone below the webcam and even on the webcam, talking loudly, and darting their eyes all over the place, they had not once been warned by the proctoring system (which is supposed to issue first warnings on any violations).

“They could have sent me a warning message to cease my inappropriate behaviour, anytime,” said the candidate.

How many people does it take to answer an NLAT simulation? 4

We have also seen a video of four people taking the NLAT simulation.

The scene: one sat in front of the laptop pretending to be interested in and working out the 10 questions they would by now know by heart, while another person sat next to them looking at their screen and worked out answers (just out of the webcam’s field of view), another person sat on the other side taking a cameraphone video of the screen, and a fourth person stood recording all three of them.

In one case, proctor kicks in, ‘crazy fun’ ensues

Another candidate said that they were deliberately speaking and playing songs while taking the simulated test with two other persons present helping them.

After a while, they received a message from the proctoring system, ordering them to stop speaking and to show the entire room on the webcam, rotating the computer 360 degrees.

So did they get busted, we asked? No, by the time the webcam was rotated one helper had darted outside the room, while the second helper hid under the table.

“That was crazy, we had fun,” said the candidate.

Other software issues

While it was admirable and necessary for the NLAT to widen the platforms usable by candidates from just Windows PCs to also include Macs, Linux and Android, that brings completely new problems with it if it is not possible to test the system on so many different operating systems.

“One of the [candidates] used Spotlight feature of her Macbook,” went one account about the built-in MacOS feature you can open that allows you to search the internet and display results right there, without opening a browser window. “She got no warning.”

That’s not to say it will be impossible to the NLAT to figure this out after the event: it is theoretically possible for a test browser window to record keystrokes and mouse movements, which could result in the system guessing that Spotlight was opened via a keyboard shortcut or mouse (though the browser probably wouldn’t be able to know what you typed into the Spotlight window).

Proctor verification messing with genuine candidates’ flow

Another candidate, who apparently didn’t try to cheat, complained that such verification took up valuable time for honest candidates.

“We have to show the room, the surroundings, verify, not move too much,” they said.

“If the verification takes long when will they do so this.”

Late / indefinite start

Coming to the avoidable, we have received several reports of candidates not having received their log-in details (or even timeslot) in time.

One candidate said: “For 12 pm reporting and 12:30 test today they send mail at 12:38. And now when we log in says test time has expired.”

“They have to ensure that everyone in one particular slot needs to get verified within 10-15 mins before the exam starts - it will be a cause of anxiety if that doesn’t happen,” said another candidate.

Update 14:01: In at least two cases we have seen, emails or SMSes arrived considerably after 1pm, in one case a message arriving after the exam had finished, nearly at 1:30pm.

But even when timeslots arrived ‘on time’ (as in, barely before the exam), that caused issues for candidates:

First of all, we got our time slots A WHOLE LOT OF 15 MINUTES before when we were supposed to appear. I was supposed to go to a relative’s house to give the exam as my laptop was not working properly but could not do that. I tried calling them to ask if i can give the simulation test and the actual exam of different devices but even they didn’t seem sure.

Now I will have to give the actual exam on a computer which could die anytime. I have heard of students getting the email at 12.34/12.40 telling them they need to appear at 12.30. Now they are not able to log in as the system says they are late.

I have sent them a couple of emails asking if there is any tech issue in between the exam, how would we reach out to them considering we are neither allowed to leave the test window nor have a phone next to us nor do they seem to have (as per the instruction manual) any internal system to solve tech issues and communicate with the proctor as there was in SLAT. They are not replying. Even during today’s test, the verification was taking too much time. i had emailed them after 25 minutes of waiting. Did not get any reply. Couple students have been waiting for almost an hour. No replies.

The timings of the actual exam are still not out yet.

Update 14:10: We understand that some candidates have begun receiving emails confirming their time slots for tomorrow’s final exam. However, some of those candidates have not received time slots for their upcoming simulated mocks yet.

What does this mean?

The simulation was supposed to be a fully proctored experience but it appears that multiple deliberate shenenigans were not flagged.

That’s not to say that those same actions would not lead to eventual disqualification in the final exam, but it does not give a lot of faith in the automated software.

We spoke to a candidate who had just sat the simulated exam and wished them luck.

“CLAT is the only one that matters,” said the candidate. “This is a joke, this is worse than Symbi [which also ran a home-proctored admissions test].” DISCLAIMER: None of the accounts mentioned herein are endorsements of cheating or describe methods that will necessarily work in the final exam, which may have more stringent security measures, in addition to potentially having a post mortem process where successful candidates’ camera footage could be examined more closely. In short, even if the system is not perfect, it is still very risky to cheat and if you’re unethical enough to try cheating in the actual final exam, no one but you bears responsibility for your failure.

Update 14:57: A new section has been added to the FAQs that confirms: “Further investigation post the examination will be carried out with help of audit logs generated on the system, and NLSIU may disqualify the candidate upon such examination”.