This post may contain affiliate links, which means we may receive a commission, at no extra cost to you, if you make a purchase through a link. Please see our full affiliate disclosure for further information.
What is the Search Engine Evaluator Exam?
The qualification test/exam conducted by Google with the help of search evaluator companies to find the next search engine evaluator is called the search engine evaluator exam.
This test weeds out all other applicants and takes only the best of the best candidates who have demonstrated their potential ability for rating by scoring a certain percentage in the exam.
This grueling test serves as a barrier between the deserving and the non-deserving candidates and between the hard-working and not-so-hard-working candidates.
Table of Contents
Who conducts this test?
The exam is conducted by the vendors on behalf of the client. The exam questions are set by the client themselves but the logistics are handled by the companies.
Which companies use this test?
This qualification test is used by the following companies-
- Telus
- WeLocalize
- RWS
to hire Raters on behalf of their client
Different names for this qualification test
The evaluator companies like to confuse us by assigning different names to the search engine evaluator job.
Likewise, they like to confuse us by naming this test differently in different countries although it is the same thing!
Telus likes to call it as
- Internet Safety Evaluator Test
- Personalized Internet Assessor Test
- Rater Qualification Test
- Web Search Evaluator Test, and so on
WeLocalize likes to call it as
- Scout Search Quality Test
RWS likes to call it as
- Search Engine Evaluator Test
Whatever the name, they all mean the same damn thing!
What is the need for this rater test?
Every day hundreds of people apply to work as a search engine evaluator with the above evaluator companies.
There has to be a way to find the most capable person who is attentive to minute details, who can follow instructions, and who keeps in mind all the KPIs and parameters, before actually rating in real life.
Hence this tough test is conducted to trim down the applicants and select the best ones that get through.
When is this test conducted?
From my experience of 8 years working in this industry, I can confidently say that this test is conducted almost all year round.
I am not saying that it is conducted every other month. But I am willing to bet that it is conducted at least 3 – 4 times a year.
But it all depends on their client’s (Google) requirements. If they need more people for a particular country, then the evaluator companies will help conduct more number of tests there.
If they no longer require raters in a particular country, then the evaluator companies won’t conduct any tests there.
How tough is this qualification test?
This is a grueling test and one of the toughest exams that I’ve appeared in my life!
I failed in the first attempt, but somehow got through in the second (consider yourself very lucky if you are awarded a second attempt these days – the number of applicants is way too high to warrant giving a second chance to candidates).
Unlike school exams, where we just need to score 30 – 45% to pass the test, here you need to score at least 80% overall to even qualify!
How soon do we get the result?
There are three parts to this test-
- For the first (theory) part, you get the result immediately (The passing score is around 85%)
- For the second (practical) part too, you get the test result almost instantly. (The passing score is around 75%)
- For the final part, you won’t get the result immediately. It takes some time. I suspect the last part is examined manually while the first two parts aren’t. (The passing score is around 75%)
Do we get to know the scores?
Never!
Think about it, if you are an employer and you hire hundreds of people every year or month, would you be comfortable or even have time to give out scores for each person while also showing where they’ve made the mistakes?
No right?
Doing so would simply make your life difficult. The same goes for this test. Your actual scores are never revealed, and rightfully so.
What is the passing percentage?
I’ve been blogging about this search evaluator job since December 2016. You can check my archives if you want to verify.
In my almost 3 years of blogging (and 3+ years of working as an evaluator), I’ve referred thousands of people to Appen (formerly Leapforce).
But hardly a few hundred have gone on to crack this test and work as an evaluator.
I did quick math and found out that hardly 1/100 people pass this test. It is based on my math, backed up by solid data that I have.
So the passing percentage is hardly 1%.
Do we get paid for the time spent studying for the test?
Did you get paid to study in college? I guess you took student loans to pay them instead.
You got into debt to get a degree that won’t or didn’t get you a decent job.
So please don’t complain!
What is the exam based on?
The exam is completely based on Google-provided General Guidelines.
The Guideline is the Bible of search engine evaluator work. And it is the only material that you will be supplied for the test, it is the only thing that you need.
Do I have to be a technical person to appear for the test and work as an evaluator?
Not at all!
You don’t need to be a technical person, you don’t need to do math calculations, code programs, or use complex software.
You should just be comfortable using a smartphone (iOS or Android) and a desktop/laptop (Windows, Mac, or Chromebook).
Search Engine Evaluator/Internet Analyst/Rater Exam Strategy for Telus, WeLocalize, and RWS
As soon as you get the General Guidelines from the evaluator company you apply to, read it like hell. You get ten days to read and complete the test. Click on every link given (I mean it) and make your own notes.
Go through every screenshot example, there are hundreds and hundreds of them.
Take special notice of minute but subtle differences in the examples. Even though the Guideline is only 168 pages long, due to hundreds of examples with links (which you have to click and study), it takes a lot of time to soak in that much information.
Here is how you should allocate your time for each part of the rater qualification test :
- Theoretical Part:Â Take 1 day aka 24 hours to complete this test. This is an open book Multiple Choice Question type test. Questions are directly taken from the Guidelines. As soon as you hit Submit, you will get an email saying that you have passed the test (or failed the test). Even though it is an open-book test, don’t take it lightly. Many people fail at this part of the test.
- Practical Part I: Allot 2 and a half days for this test. This has to be completed on a PC or a Mac or Linux running Chrome or Chromebook. This is a tough exam and hence be very diligent while giving the scores. Your result will be notified immediately as soon as you hit the submit button.
- Practical Part II: Allot 3 and half days for this test. You’ll need a mobile device (not a tablet) and a PC or a Mac to complete this test. This is the toughest of all the exams (at least it was for me).
Other helpful tips for this test
Don’t leave anything to the last day
We all tend to leave everything to the last day and rush through. I am guilty of it and probably you are too. This is not the kind of exam that you can rush through. Please follow the time management strategy detailed above. Thank me later.
Take Notes
As you read the guidelines, make sure to go through all the examples and take notes. Take note of the minute but extremely important difference between different kinds of rating examples and types.
Refer to the guidelines all the time
Always go through the guidelines before attempting any questions in the test. Remember this is an open-book test. If you are stuck on a question, refer back to the guidelines and make sure that you find that page/article/example that is kind of similar to the question that you are attempting.
Control +F or Command+F is your best friend
When referring back to the guidelines, always use “Control+F” in Windows or “Command+F” in Mac to look for the keywords contained in that question.
Protip
While appearing for the test, you don’t have to complete every question of that part in one sitting. This applies to all parts of the exam.
Finish a question ( which may have multiple sub-questions), then hit “submit and stop”, take a break, and restart the test again.
You have 90 mins to complete each set of questions ( for all parts of the exam) before the question expires.
Search Engine Evaluator Exam Free Course
Want a free 5-day course on search engine evaluator exam?
Enter your name and email in the form below
and get instant access
Hi Sam,
Great work on the blog,
I have question , Do i really need to keep track of time for the invoice or i can just say for if i do 5 task of 3mins each i can just mention 15mins straight. because even if i exceed the time limit i will be paid for the same 15mins. should i just keep track of my task counts. and i will make sure i spend at-least 3mins on each with my research.
You cannot bill more AET of each task. If the tasks is for 5 mins, if you submit it in 3 mins, you will bill for 3 mins only. If you take 7 mins, you will bill it only for 5 mins, as the task is for 5 mins. This sort of questions are best answered in social feed. 🙂
Hi Sam,
Your blog is real help.
I cleared my test on first attempt(Lucky).
Would recommend others.
Cheers,
Glad to know. Cheers
3
Hi Sam Bro
My Mail Say,
”Your exam will be due on Thursday, December 13, 2018 at 12:00am (PST)” ,
So i have till Friday 1.30pm IST., to complete my test right?
Hello, Sam. Nice article, but I applied for the job and opened a bank account. She told me to visit a link which I did and applied for credit. I didn’t qualify for the credit because my credit score was too low. She did say however that I was employed and when I receive the new debit card that I would get paid for what I did last week. My question is why do they need my bank login information to send me money? I’m not sure this is really legit! What can you tell me about this? Can I truly trust them not to take money out of my account if I give them the login info?
You don’t need a good credit score to apply for this job. And they will never ever tell you to apply for credit or debit card before even proceeding with qualification process. And they don’t need your bank account log in details. Please change your account details immediately. You have been scammed.
Genuine Leapforce or Appen recruiters would never ask you to do all that and they wouldn’t in million years ask you for bank account log in details. Her is their official website where should be applying –
https://connect.appen.com/qrp/public/jobs/list?uref=6c2ff9f87034def5768bd6b5bb65d9df
Leapforce is now known as “appen” now as they have been acquired but it. don’t be confused when you see “Appen”
Do they actually give us 7days to study and then to take exam in the next 7days,
My mail says ‘You are scheduled to begin the qualification exam on Friday, November 16, 2018.’ ‘You will have seven days to complete the exam. Your exam will be due on Friday, November 23, 2018 at 12:00am (PST).’
?
I don’t remember so well. I think I got 7 days to study and another 7 days to complete the exam. But I feel like they don’t give study time any more. Just 7 days to study as well as appear for the exam.
hi ky ,
me to have the same email can you help me
Hi Me,
i am taking lionbridge’s exam. and i cant help. Study the guidelines for any exam, thats what you need.
Hi, I have my exam by the end of this week. My email ID is mounikappen@gmail.com. Could you please send me any helpful link. Also how the test? Did you qualify it? I really need your help. I have my test in a week and I don’t get it.
All that you need to pass this test is the guidelines. Please go through it thoroughly. Don’t rush and don’t keep anything for the last moment. Its a tough exam that requires dedication.
Hey!
Thanks for all the help!
Just wondering, on landing page purpose, should you be specific i.e “purpose is to share a news article about polarbears” or should you think about the website purpose i.e “to share and inform of the latest news”?
Thanks a bunch
Depends.
Hi, but is it possible that i have no chance to practice a little bit with the pratical part, before taking the exam ?
Kind regards
Unfortunately no. Good Luck. But if your concepts are clear, you should be able to appear for it.
The theoretical test questions say: Select all that apply. I’m not sure if we also need to mark the ones that don’t apply “False”
The instruction should say “Mark all that apply TRUE and all that do not apply FALSE. Is this correct??????
Before “select all that apply” there is a question. Pay close attention to the wording of that question. Good Luck.
Hello Sam, you did a great work with this article. Please is there a way of knowing if/when a rating question times out?
I’m asking cos after rating the second question in part 2, clicking “submit” took me back to the raterhub start page instead of the next question. Is this normal cos I answered the question within 30mins.
Question ( along with the sub-questions) times out in 90 mins. It shouldn’t time out in 30 mins.
Hi Sam… I watched a video on youtube where someone said that you got to have a bachelors degree to qualify for jobs@Leapforce. Could you please tell me is it true? Or anyone with basic internet and social media knowledge can qualify for the job ofcourse after passing their exam.
Companies like Leapforce aren’t so strict about that requirement. They do asks for it though. If you are pursuing graduation, you can still apply for it.
Hi Sam,
I have received an email from leapforce stating to agree to some terms and conditions. After all, when I select all projects option there I can see Nile (project) 3.50/hr followed by qualify tab. Now I’m afraid of clicking on qualify tab here since I don’t have any idea of proceeding for the exam. I didn’t receive any exam material or time frame for the exam. As im noob here how can I proceed with the examination? Please guide me. Thanks
Once you click qualify, you will receive the exam materials.
And Nile test isn’t that tough. Good Luck. 🙂
hey man I failed part 3 gonna try it again. the second time is harder ? I mean the 3 parts. Can you help with some doubts that I have ?
I don’t help with the specific answers. I can help you with general overview, but not with answers to questions.
Hi sam , i came across your content and this is great artical. I am new in this Area so can i do sme or se job ? If yeas how ?. which site accept my application quickly ? Plz reply .
You can do both.
Leapforce has kind of slowed down in processing the applications after being acquired by Appen. And Lionbridge is hiring for Online Map Analyst. Appen processes the application a bit fast but their pa scale is $3.5/hr. They have also reduced the for Leapforce new workers, not for us luckily.So it’s really your choice where you want to go.
sam,there are quality controllers for us,my question is how to be quality controller? were they also raters like us? later got promoted?
We also got senior raters who evaluates our work. But they are all from USA and they do get promoted with higher hourly rates. They are called Preferred Raters, not sure about Lionbridge though. They gets promoted after consistent high quality works. I think your quality head Aswhin would be the best person to answer that question.
meaning of consistent high quality works?
Meaning raters would maintain at least 90-95% accuracy in reviews for a long period of time.
I alsoso failed in my first attempt,I cleared the exam in the second attempt.
The exam is pretty tough. That’s all I can say. 🙂
Yes,It takes time to understand the guidelines
hey joydip how you clear part2 of the exam.can you eleborate something
Just follow the guidelines.
How many questions..is it tough like JEE ENTRANCE EXAM also are all mcq?
Its pretty tough but doable.