This post may contain affiliate links, which means we may receive a commission, at no extra cost to you, if you make a purchase through a link. Please see our full affiliate disclosure for further information.
Web Search Evaluator
This post will tell you anything and everything you need to know about the web search evaluator job.
I will lay threadbare all the inside details about this job that is not available online. I am an active evaluator and hence I know everything about this job.
If you are a stay at home mama, a college student, an introvert, jobless or anyone who is looking to work online, then this is by far, the best online job you will ever find in the entire world wide web.
And it is much more stable and consistent than survey sites.
Find out what it is and how to work as one –
Table of Contents
What is Web Search Evaluator aka Search Engine Evaluation?
Web Search Evaluator Job Description
Web Search Evaluation aka Search Engine evaluation is a process of evaluating and rating websites and web-pages based on the General Guidelines given by Google or Microsoft. It involves finding out the quality of websites and how relevant or helpful a result is for the queries issued by user.
Look at this image below –
A user (like you and me) issued a query world war ii in his/her mobile device. And he/she got the result as shown above. (Other results below are truncated due to lack of space)
The result is pretty helpful to a user issuing the query world war ii
Go ahead, type something similar if you feel like.
Now, in search engine evaluation, we have to rate how relevant or useful a particular result is to a given query.
Results could be web-pages, websites, YouTube videos, apps, pdf files, songs, and so on. We also got to evaluate the quality of web-pages or websites.
So in short, a web search evaluation aka search engine evaluation is a process of evaluating websites and web-pages based on the parameters outlined in General Guidelines of Google ( or Microsoft) and also finding out how relevant or useful a particular result is for a given query.
Due to NDA I signed with them, I cannot disclose any specific tasks that we do inside.
Terminologies simplified
There are different terms used for people who work as a search engine evaluator or web search evaluator or Google Rater. They are also known as internet analyst, raters and so on. They all mean the same thing.
Here are four different terms with which they are known –
- Web Search Evaluator
- Search Engine Evaluator
- Personalized Internet Assessor
- Internet Safety Analyst
- Internet Analyst
- Google Rater
- Rater
I will be using these terms interchangeably here. They will all mean the same thing. Please don’t get confused.
Different companies uses different terms for them
Appen (previously Leapforce) likes to call them internet analyst.
Raterlabs likes to call them internet analyst
Lionbridge calls it personalized internet assessors. They also refer to it as raters or as internet safety analyst
But they all mean the same thing – Google Rater, search engine evaluator aka web search evaluator.
What do Search Engine Evaluators do?
A search engine evaluator evaluates web-pages and websites and judges its quality. However, a search engine evaluators main work is to rate the relevance and helpfulness of a result to a query issued by a user.
How to work as search engine evaluators?
The process involves applying to the companies mentioned below and waiting for them to get back to you. It can take anywhere from few days to few months. If they don’t get back to you within 6 months, you may re-apply again.
But till its 6 months, do not re-apply with different email accounts because in your first application itself, your IP address would be captured. Re-applying again with different emails would only send you back to the waiting queue.
Initial Requirements Checklist.
- Have in-depth, up-to-date familiarity with social culture, media, and web culture
- Excellent comprehension and written communication skills in English
- Broad range of interests, with specific areas of expertise a plus
- University degree or equivalent experience (degrees in-progress are acceptable). Advanced degrees a plus. (In my opinion, it is not really required. Just apply for the positions available).
- Excellent web research skills and analytical abilities.
- Ability to work independently with minimal supervision
- Possess a high-speed internet connection (Cable Modem, Fiber, DSL, etc.)
- Use of an Android phone version 4.1 or higher, Windows phone version 8.1 or higher, or an iPhone version 4s or higher
Plus, you have to be Googly enough :). I mean, you have to be comfortable using Google to research stuffs, look for things and be comfortable looking around a bit.
Where to apply and how to apply?
Three companies are hiring at the moment –
- Appen/Leapforce (I work here)
- Lionbridge
- Raterlabs (Only for Americans)
- iSoftstone
If you want an in-depth comparison of the 4 evaluator companies, read this article HERE
Applying to these companies is pretty straight forward. Just click on the links above, you are taken to an application page, you just need to fill in your basic details, upload CV and bamm ! You are done.
Lionbridge application process is a bit lengthy though. You need to fill in all your details, academic, work experience and others, and upload CV as well.
{ NB – Leapforce was an independent company founded by a former Google employee. It split in to two – Leapforce and Raterlabs few years back. Raterlabs was created exclusively to hire American search engine evaluators. Now Appen acquired Leapforce and hence both Leapforce and Raterlabs have now became part of Appen. That’s why you would see Appen logo and branding in the previous Leapforce website.
Also you will notice that Raterlabs website design and everything is same as current Appen/Leapforce account.
Also note that the previous Leapforce website/company is also known as Appen Connect}
Who exactly will you be working for?
Although, you will be working for the aforementioned companies, the real client you will be working for is either Google or Microsoft.
In most cases it is Google.
Google contracts these companies mentioned above, and they in turn look for raters like you and me.
So, although you won’t actually be Google employee, you will indirectly be working for Google in making the world wide web a better place.
How to write CV for search engine evaluator position?
Now writing a good CV is pretty important to be considered for this position. Thousands of people apply to these companies every month. Standing out from them is pretty tough.
Here is my take on how to write a good CV.
- Make it single page.
- No need of cover letter.
- No need to attach your photo.
- Make sure to not leave any gap years in your academic, professional details
How long does it take to hear back from them?
It can take anywhere from few days to few months. It all depends on their needs and demand from client.
People often complain that they are yet to hear back from them. Often times, the emails from Appen or Lionbridge ends up in spam folder. And when you get an email from them, the company name “Appen or Lionbridge are not immediately visible on mobile devices.
Where do we mostly check our emails these days? Mobile phones !
Take a note of these emails and make sure to check your spam as well once you apply to these companies
Here are the emails they could email you from –
Appen/Leapforce email addresses –
- agent-qualification@connect-mail.appen.com
- support@connect-mail.appen.com
Lionbridge email addresses –
- ratingexam.support.bal@lionbridge.com
- lionbridge.exams@gmail.com
- rating.program.bal@lionbridge.com
- community.bal@lionbridge.com
Raterlabs –
I don’t work here or don’t know of any one who works here. So you will have to check your email including spam and other folders regularly if you apply here. And look out for the keywords like qualification test, raterlabs, and so on.
Note that you won’t see the word Appen, Lionbridge or Raterlabs in the email preview unless you click and double check. So be a bit diligent in checking your emails after applying to these companies.
Where to work as one?
Now this is a tough one to decide. Here is a comparison table –
| | | |
|---|---|---|
| Hires worldwide for all positions except Americans who wants to work as search engine evaluator. | Hires worldwide for all positions except Americans who wants to work as search engine evaluator. | Only for Americans |
| One can work for unlimited number of hours except for Brazilians who are limited to 10-20 hours per week. | Their hours are limited, ranging from 20 – 30 hours per week. They give bonus hours of up to 15 hours per week when the task load is high. | Fixed 26 hours per week |
| Apart from search engine evaluator position one can work in multiple projects at the same time. | Has multiple projects but one person can only work at one project at a time. | Only search engine evaluator project is available. |
| Candidates are hired as independent contractors for 6 months. | Candidates are hired as independent contractors for 12 months. | Hired as part-time employees. |
| Pay rate ranges from $3/hr to $13 per hour. Indians are paid $3/hr, Americans are paid up to $11/hr while Europeans are paid up to $13/hr | Pay rate ranges from $6.81/hr to $13 per hour. Indians are paid $6.81/hr, Americans are paid up to $12/hr while Europeans are paid up to $13/hr | $11 - $13/ hr depending on which state you are. |
| Pays $25 per referral. | No referral program | They do have a referral program |
| Paid via Payoneer under NET 15 days terms | Paid via direct bank transfer under NET 45 days terms | Paid via Payoneer/direct transfer under NET 15 days term |
Do they hire in my country?
Well, they hire from almost any country in the world. You will have to check here whether they are hiring in your country or not.
They don’t hire from most African countries, Pakistan, Nepal, Bhutan, Sri Lanka, Bangladesh.
Do check it out here whether your country and language is in the list or not –
- Appen/Leapforce
- Lionbridge
- Raterlabs (Only Americans)
- iSoftstone
How much can you earn with search engine evaluator job?
You can earn anywhere from $500 – $3800 easily, depending on where you are located. Hey I made $2007 myself, being an Indian.
If you are an American working as a part-time employee who is limited to 26 hours per week, you can make up to
26*4*$11-$13 = $1144 – $1352
Brazilians used to be limited to 20 hours per week. But now since Appen brought in a lot of projects, there isn’t any weekly limit that they got to adhere to.
But everyone else can work as many hours as you want.
How much do they pay you?
As mentioned in the table above, payments ranges anywhere from $3 per hour to $13 per hour depending on where you are located.
Projects available inside?
All the companies got multiple projects inside except for Raterlabs.
Raterlabs is the only company that hires Americans for search engine evaluator position and it only has this one position open. Sure its website mentions openings for multiple countries with several projects. But trust me, its only for Americans and there is just one project available – Search Engine Evaluator aka Internet Analyst.
Appen has multiple projects apart from this. They offer Facebook projects (social media evaluator), Instagram projects and many others. Can Americans work in this projects? Yes, except that they won’t get the evaluator position to work for.
One can work for multiple projects at the same time in Appen.
Anyone who joins Appen after April 2019, and is selected to work as a search engine evaluator aka web search evaluator is eligible for $250 bonus. $150 for working at least 20 hours in the first two weeks and another $100 for working at least 20 hours in the next two weeks.
Lionbridge as multiple positions as well like Maps (Apple), Social media evaluator and so on. One cannot work for multiple projects at the same time.
One can also not work for different companies in the same project at the same time.
How to get paid?
If you want to work for Appen or Raterlabs, you need a Payoneer account. Here is a link to create one. You will get $25 bonus if you sign up via my link. You will get that bonus as soon as you are paid $1000 to your Payoneer account.
What about the qualification test?
There is a tough qualification test before you can work as search engine evaluator. It is one of the toughest exams that I have written in my life.
Hardly 1 out of 100 people gets through it i.e. only 1% qualifies for this position.
It is not a surprise considering the fact that the passing score is 87%
The qualification test consists of 3 parts –
- Theoretical part
- Practical part I
- Practical Part II
How to pass the search engine evaluator qualification test?
Before appearing for the test, you will get a 164 page General Guidelines to study. It will be your Bible, before and after you pass the test.
As soon as you get it, read it like hell. You will need at least 20 hours to cover all of the concepts and go through all the examples.
Here is my recommended time distribution for the three part qualification test –
- 1 and 1/2 day for the Theory part.
- 2 and 1/2 day for Practical I
- 3 and 1/2 day for Practical II
If you fail the test, you may get a one time retake.
Why people fail at search engine evaluator qualification test?
Most of the people fail at the third part of the exam. It is the toughest of all the parts. Some people fail even at the first part.
There are many reasons why people fail.
- They don’t study enough or can’t study enough.
- They don’t give much time or don’t have much time understanding the concepts.
- They rush things through.
- No attention to minute difference between different types of rating.
To help people crack this test, I build a course, two courses in fact –
One is for complete newbies who got no idea what search engine evaluation is.
The other one is for those who failed the qualification test . You can check it out here –
Course
Thousands apply for this positions and getting a retake chance is getting harder and harder. So don’t take risk. Enroll in one of the courses above. Learn from someone who has been at this game for almost 3 years now. Learn the top 4 reasons why people fail and how to avoid those mistakes.
Final words
I have been working as Search Engine Evaluator for almost 3 years now. It has benefited me tremendously. It has helped me in many ways –
- I can figure out fake news very easily.
- I now understand what is SEO and why certain pages rank high in Google search while others don’t.
- I take every online info with a handful of salt.
According to me, this is the best work at home job for moms, students, introverts and for anyone who likes to work from home.
Remember, you will be working for Google indirectly. And you will be working to make the internet a better place for all.
So, go ahead, make your decision and sign up in these three companies –
- Appen/Leapforce (I work here. It has $250 bonus as discussed above)
- Lionbridge
- Raterlabs(Only for Americans)
- iSoftstone
You also need a Payoneer account to get paid, so sign up here and get your $25 bonus*
Hello, Sam. Great stuff bro! Kudos.
I had a question. For the social media evaluator post, does one require to have a real fb account?
I never used social media as I was wary of giving out personal information and posting pics. I have a fake account though which I have been using for years. Can I use that ? Or do you need to link your fb account to them in some way? Last year I did do a social media related task where I had to note what sort of posts my friends had made in the recent past. If I remember correctly they didn’t ask me for a link to my profile. I got paid at the end of the task.
Hello, Sam. I had another question. Appen has significantly reduced to 3$ their hourly rate. How much can one expect to make from the rater job in a month in Indian rupees on average. I ask because I read that you don’t have tasks available at all times.
Due to COVID-19, task flow had been very less in the month of April. This month is much better. One can easily average 100 – 160 hours in normal times.
In social media evaluator job, you would be rating from inside your FB profile. You don’t need to have your real name in the Facebook profile. It can be anything. And yes you would need to link your FB profile to their platform, to be more precise, they would need to add your profile to the rating tool.
So, I can link my fake account and it won’t make the difference? i mean won’t it be an issue when they see the names don’t match? lol
Oh and for the rater job, which one gets your vote? Appen or Lionbridge? You have been an old employee but the new ones will get a much reduced pay on Appen. So, taking that into account whom would you bat for?
This post will answer your query – http://outsidethatcubicle.com/leapforce-vs-lionbridge-vs-raterlabs/
Not a fake account. Your real Facebook account under a fake name works fine.
Real fb account with a fake name? I don’t get it. Isn’t an account with a fake name on fb a fake? Can u elaborate? I have already read that article. I just wanted to know your personal opinion. haha
Teemwork (earlier isoftone) pays 5$? They hire from India? How are they in comparison to Appen and Lionbridge? Appen pays much less but is there a better probability of getting tasks with them? Also, you say with Appen, one can get 100+ hrs of work. So, that means one can easily make 300$+?
Sign up with whatever Facebook account you have that you use regularly. You should be fine. Name on it doesn’t matter. My personal opinion is summed up in that article 🙂 Good Luck.
Pingback: Make Money Online with Web Search Evaluator Jobs - This Mama Blogs
Olá Sam, tudo bem?
Sou do Brasil e não falo inglês ainda, existe alguma forma de trabalhar para as empresas sem ter o inglês fluente? Existem empresas onde não seja necessário o inglês, e possamos utilizar mecanismos de tradução? Consiredo ter as demais habilidades necessárias para o trabalho.
Desde já agradeço, até logo mais.
Hola. I don’t speak Spanish bro. Could you type in English please?
Hi Sam, how are you?
I’m from Brazil and I still don’t speak English, is there any way to work for companies without being fluent in English? Are there companies in which English is not necessary and can use the translation mechanism? I consider having the maximum skills necessary for the job.
Thank you in advance, see you later.
Si. Yes! You can work in your own language. Look these companies hire in the language of your country. In Japan, you can work in Japanese and in Brazil, you should be able to work in Portuguese. Just go ahead and apply. Good Luck Léo 🙂
Ok Sam, thank you guy.
Welcome 🙂
Hi Sam,
I had applied on Appen and received a mail from them asking me to digitally sign some documents on their site. After signing the document, I am taken to a page which shows a list of projects with a “Qualify” link against all the projects. What should I do here? Do I need to qualify for all these projects individually or do I need to select only one. And which one is to be selected if I select only one project?
Thanks
You will need to select the project called “Yukon” if you want to work as a search engine evaluator. Yes, you need to hit “Qualify” button and qualify for each project individually.
If you’ve worked for both Google and Microsoft companies through the hirers, were the tasks pretty much the same for Microsoft as with Google? I’m wondering is one better than the other overall or more interesting than the other?
Thanks
I worked as an ad evaluator with iSoftstone,now known as Teemwork.ai. I worked for like a month or two. I didn’t have time to handle both. Tasks were about Bing ads (Microsoft project). For Appen, I work as a search engine evaluator where we evaluate search results returned by Google. So we can’t really compare the two.
Hi SAM,
I am Prakash from Hyderabad India.
I need help for the maintain the quality for long time. Please suggest me brother.
I will wait for your valuable reply.
Read the guidelines, practice the simulations.. that’s it.
Hey Sam,
Nice blog. I was working on a project in Appen till Dec 25, 2019, worked for it almost 4-5 months and one day suddenly they removed me from the project. I found that many people were removed without notice. I thought maybe it was because of quality, but later they sent an email telling me that it was not because of the quality, it was because the project ended. My question is are all projects are like this? Can they remove us anytime? Have you ever faced such situation there?
Not all projects are long term and not all projects are short term. Evaluator jobs like Yukon, Bane, Sanaga, Wouri are long term. Social media evaluator projects like Nile, Tahoe, and some one time projects are short term.
Yes I was removed from Nile within 12 months. Happens. Just be sure to be in long term projects.
Hi Sam,
Have you qualified Tahoe qualification quiz. I need your help. I am not able to qualify on it’s first attempt.
If yes then do let me know at pallaviankurdixit@gmail.com
No I haven’t. I think they give you two chances. So study hard and good luck. I heard that its test is not so tough.
Give me the link of the social feed
It is inside Appen dashboard. Look for a tab called “Social” and you should be able to ask questions there.
Hi SAM, I have recently qualified for a project in Appen. But I can’t start working as it is required SRT login to the workplace that I am unable to do as I don’t know what is my SRT email. Please tell me how to get it? I have sent a mail to the support team but not received any reply yet.
Could you post in the social feed of Appen please? I haven’t worked in a project that requires SRT log in years now. It used to be different long back. Post in the feed please, you should get correct guidance inside.
That sums up everything. And I am not Appen support who knows better right? They know it better. Wait for some time, I am sure they will send out SRT invite as soon as they need additional raters. I am deleting your above message as it contains sensitive info.
ok delete it